How to Innovate through Successful Experimentation
To promote innovation in their organizations, intrapreneurs need to not only have brilliant ideas, but also run experimentation to test those ideas. Experimentation is designed to answer just one question: As we consider a new venture, what would be the cheapest, fastest, and least risky way to learn whether this new venture is worth the investment and implementation?
Experimentation involves three crucial aspects — its finance and objectives, its design and result evaluation, as well as its execution.
How to finance experimentation and what are the objectives?
Finance:
Experimenting requires resources and investment. Usually, there are two types of budgets in an organization — project budget and operation budget, but no experimentation budget. Experiments are rather different from both projects and operations — the time it takes to deliver result, the velocity of decision making, and the way to assess result are all not the same — thus we need to allocate specific funding for experimentation alone.
One way of doing that is to assign 2 to 5 percent of the fund from project budget to a venture portfolio for experimentation, as most organizations do. Why from project budget? Because usually they share similar resources such as people, equipment, and so on — although this venture portfolio is managed differently from a project portfolio. A venture portfolio can be viewed as an incubator of startups, which is managed by an agile leader assuming the role of an intrapreneur. Out of, for example, 10 ventures in such a portfolio, maybe just one could be wildly successful like a star, another one or two would only reach break even, and the rest seven or eight wouldn’t give birth to any innovative solution at all. When we evaluate these ventures or experiments, we can’t look at the result of each separately; rather, we need to look at the whole portfolio, knowing that the stunning success of this single star experiment is built upon the learnings from those others who didn’t succeed in the same way, and that there is no shortcut to identify successful ventures before learning from failure.
Objectives:
To evaluate the performance of the venture portfolio is to know whether the objectives of an organization’s intrapreneurship or innovation are met. There are three objectives in general.
- The first one is obviously to have experiments that succeed and give birth to actual innovative solutions that could bring a meteoric rise or a spectacular boom to the business.
- The second one is learning, which is also the key to achieving the first one. Although learning is not an ROI type of KPI, it still can and has to be measured. Some techniques like innovation accounting are precisely designed to measure learning and progress in experimentation.
- The third objective, also one of the reasons to develop intrapreneurship in the first place, is to engage workers. Ideas don’t come just from executives; brilliant ones can come from everywhere in an organization and involve any employee, so does the experimentation of those ideas.
How to design an experiment and evaluate the result?
Design:
A value hypothesis is something formulated to express what value is expected to derive from a potential solution that is not yet implemented. The goal of experimentation is to validate or invalidate value hypotheses by testing them. How? Here is a concrete example.
A public transport operation company faces the recurring problem of having frequent disruption and delay of the trains. One potential solution is to install an electronic service status display at the entrance of each metro station, so that people can see whether the operation is disturbed or not when they arrive, and react accordingly. Implementing such a solution can be very expensive and certainly involves risks since no one knows to what extent people will find it useful. That means, the potential solution needs to be tested in a way that is fast, cheap, and with minimum risk.
Experimentation is designed to have usually one or more value hypotheses tested so that we can understand whether the hypotheses will really deliver the assumed value. In our example, one testable hypothesis is to install a non-electronic board for service status announcement in front of three metro stations for one month’s time. When there is an incident affecting these metro stations, at least 50 users will be surveyed with one question: Is this display useful to you or not? The hypothesis will be considered as validated if at least 60 percent of the respondents answer “Yes,” and invalidated otherwise.
In this example, the test of the value hypothesis involves responses from actual users based on their experience with a product for trial (the non-electronic board). It’s not an analysis based on a document or abstract questions asked over the phone of random users. Likewise, all the experiments need to be designed with a real prototype and feedback from actual users. Or else, they are worthless and can’t lead to any solid conclusion.
Evaluate:
To evaluate the result after an experiment is done, we have to understand that the way to define success of an experiment is quite different from that of a project. Whether we validate or invalidate a hypothesis in an experiment, it’s a success in both ways. If the hypothesis is validated, it may turn into a promising product and generate value at scale; if it’s invalidated, not only a precious lesson can be drawn but a potentially huge mistake of investing in something that will not yield the expected benefits can be avoided. The only way to fail an experiment is when we don’t know whether the hypotheses are valid or not. The role of an agile leader or an intrapreneur is not only to embrace, support, and promote experimentation, but more importantly, to change people’s way of defining what is failure and what is not, which is an inseparable part of an organization’s culture.
How to execute an experiment?
To execute experiments, we need to consider the bite-sized nature of the employed resources to enforce the high velocity of the experiments. We often use the word “box” to refer to the thoughtful allocation of resources to experimentation, and there are four types of boxes that work together to make experimentation a unique way of working and a distinctive type of experience unlike any other:
§ The budget box: As the name implies, when we start an experiment, we only have a fixed amount of money to do it. There will not be one more dollar available, and if we cannot do it within this budget, we just don’t do it at all. The idea can be big, but it has to be split into small experiments, each fitting into a budget box.
§ The time box: Speed is of the essence when we’re innovating, so we need to set up a maximum time frame to finish experimentation, for example one month, two month, etc. The faster our reaction to the changes in the market, the more value we can deliver. If we are too slow, even the most ground-breaking idea will yield nothing but waste.
§ The team box: Team box here refers to the high level of team dedication it takes to execute experiments — dedication of the team members is as important as their expertise to succeed in experimentation. Without a fully committed team, no effective collaboration or real accountability can be achieved, thus no result can be generated from any experiment.
§ The location box: Sharing the same location can help people to collaborate spontaneously. If people work in different buildings and meet occasionally here and there, they can’t produce anything together effectively. It’s much better to have a virtual or physical meeting room explicitly allocated for experiments.
As crucial as the four boxes, another must-be-met condition for the successful execution of an experiment is that the team has to be autonomous. It should be a team of intrapreneurs who have the freedom of making decisions by themselves. The role of experts and a manager or a leader on the team is merely to “serve” the team members by supporting them and offering guidance instead of controlling or directing them.
Last but not least, many well executed experiments are done with partners such as an external expert, an out-of-the-company startup, or some academics. They can all contribute enormously to the experiments that have never been done before in the organization — through challenging its habitual way of working or shaking up its old systems.
Related articles and book:
Four Enablers for Scale-Up of Organizational Innovation
How to Generate Ideas and Qualify Them for Innovation?
3D Agile Leader Model — Introduction of the Three Dimensions
3D Agile Leader Model — Intrapreneur: Ideation
Agile Leadership Explained: We Can All Be Agile Leaders and Change the World Together! (Kobo)
Agile Leadership Explained: We Can All Be Agile Leaders and Change the World Together! (Amazon)
Sources:
Collet, B. (2019). Agile leadership. (Online course). https://www.udemy.com/course/agile-leadership/
