In this chapter we turn to the most neglected side of innovation management: its effect on those who were never asked about it. We propose a framework that builds responsibility into the initiative journey, and from there move on to the ethics of experimenting on people, the governance of data and artificial intelligence, fairness in the ownership of ideas, and the environmental impact of innovation.
A city launched an app that automatically detects road defects from residents’ phones while they drive and ranks maintenance work by the number of reports from each neighborhood. The app looked like a clear success: thousands of reports, faster response, and lower costs than field inspections. A year later, an engineer noticed that maintenance had become concentrated in newer, higher-income neighborhoods, while it had declined in older ones. The reason was that residents of the older neighborhoods, many of them elderly, used the app less. The app had redistributed the service without anyone deciding that it should.
There was no intended injustice in this case. The app was designed in good faith and succeeded on every measure applied to it. Yet the measure itself, the number of reports, reflected who used the app, while those who needed maintenance remained outside the picture. This paradox is our starting point for the question that occupies this chapter: how can responsibility be built into innovation management as part of its quality?
After studying this chapter, the reader should be able to:
As early as the 1930s, the sociologist Merton pointed out that every purposive social action produces consequences its actor did not intend, owing to incomplete knowledge, errors of judgment, and a focus on immediate rather than distant results (Merton, 1936). Innovation, by its very nature, is action in unknown territory, so its unintended consequences are all the more likely.
One of the clearest examples is what Obermeyer and colleagues uncovered in an algorithm widely used in the US health system to identify patients in need of additional care. The algorithm used past treatment costs as a proxy measure for health need, and because less was spent on treating Black patients, for reasons related to access to care, it underestimated their need. The researchers found that correcting the measure would have raised the share of Black patients receiving additional care from about 18 percent to about 46 percent (Obermeyer et al., 2019). This case is another illustration of the point made in Chapter 16 that an indicator can swallow the goal: a proxy measure carries assumptions that those who use it may not see.
Anyone who wants to control these consequences faces a dilemma formulated by Collingridge. In the early stages, an innovation is easy to change, but its consequences are not yet known. In the later stages, its consequences are clear, but changing it becomes difficult and costly because it has spread and interests and systems have become tied to it (Collingridge, 1980). Figure 19.1 illustrates this dilemma.
Change is easy and cheap, but the consequences are unknown, so no one knows what needs to change.
The consequences are clear, but change is difficult and costly, because the innovation has become bound up with systems, interests and habits.
Raise questions of responsibility early, monitor consequences during experimentation and scaling, and build the capacity to adjust before the solution hardens.
This dilemma mirrors the cost-of-change curve presented in Chapter 17: responsibility, like learning, is cheaper at the start. That is why Chapter 12 added the responsibility lens to the lenses for evaluating a solution, so that its questions are raised from the very first concept.
Stilgoe, Owen and Macnaghten proposed a framework for responsible innovation built on four dimensions that are practiced during the innovation process itself (Stilgoe et al., 2013). Figure 19.2 presents them together with the question each one raises.
What if the solution succeeds at scale? Who might be harmed?
What are we assuming without noticing? Whom are we not seeing?
Have we heard from the least represented groups?
What would make us change course? And who holds the decision?
These dimensions are consistent with what the book has built in earlier chapters. Inclusion extends the understanding of those who live the problem in Chapter 11; responsiveness extends the stopping criteria in Chapters 10 and 14; and reflexivity, that is, critical review of oneself and one’s assumptions, extends the discussion in Chapter 8 of the psychological safety that allows a team to admit what it does not know. Responsible innovation is simply good innovation management, with a wider circle of people it cares about.
In the opening case, the anticipation question alone would have been enough to reveal the problem: who does not use the app, and do they need maintenance any less than others? Inclusion would have added an alternative reporting channel, such as a phone line or a neighborhood monitor, along with regular field inspections of the neighborhoods where use was lowest.
Chapter 14 called for experimentation before scaling and cautioned that the beneficiary of a public service is not a test subject. The balance between the two rests on well-established ethical principles for research involving human beings. The Belmont Report set out three basic principles (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979):
In innovation experiments, these principles translate into practical questions. Do participants know they are in an experiment? Can they withdraw without losing the service? Are there stopping conditions if harm appears? Was the participating group chosen because it is best suited for learning, or because it is least able to object? In universities, research involving human beings passes through research ethics committees, and it is advisable to apply the same principles to student projects that test solutions with real beneficiaries.
Before any experiment with beneficiaries, ask yourself: if I, or a member of my family, were in the group to be experimented on, what would I want to know in advance? And what would I want guaranteed? Write down the answer and make it part of the experiment card.
Many innovation initiatives today depend on personal data and on artificial intelligence, both of which are sources of great value and of great risk alike. The Kingdom has established a legal framework for protecting personal data with the issuance of the Personal Data Protection Law (Personal Data Protection Law, 2021 [Arabic]), and the Saudi Data and AI Authority (SDAIA) has published AI ethics principles intended to guide the development and use of AI in ways that safeguard rights and strengthen trust (Saudi Data and AI Authority, 2023 [Arabic]). Internationally, these frameworks intersect with the OECD AI Principles, which emphasize values such as respect for human rights and fairness, transparency and explainability, robustness and safety, and accountability (OECD, 2019).
Compliance with these frameworks does not remove the need for questions the innovation team must ask itself, the most important of which are:
This section returns us to a thesis presented in Chapter 2: that artificial intelligence raises the value of the human rather than eliminating it. The larger the share of work that tools take on, the more important human judgment becomes in determining what they should do, for whom, and at what cost.
Innovation management raises questions about the ownership of ideas and solutions. Who owns an employee’s idea that the organization has developed? Who owns a student project that originated in a course or a university laboratory? What share of the value of an innovation goes to the innovator? Uncertainty about the answers harms both sides: innovators who do not know their rights may withhold their ideas or take them where they will be protected, and an organization that has not defined its policy faces disputes once an idea succeeds.
Fairness here requires a clear policy, announced in advance, that specifies what the organization owns, what remains with the innovator, how any returns are to be shared, and how credit is attributed. This connects directly with the discussion in Chapter 10 of procedural justice and attributing an idea to its originator, and with the discussion in Chapter 7 of recognition as one of the innovator’s foremost needs. In universities in particular, such a policy needs to be understandable to students and to encourage them to develop their ideas within the university’s own ecosystem.
Intellectual property also bears on the open innovation discussed in Chapter 18: an organization that makes its data available or collaborates with partners needs clear rules on what is shared, what is protected, and how individuals’ privacy is preserved in the data it releases.
The question of responsibility extends to innovation’s impact on the environment, one of its longest-lasting effects and the hardest to reverse. Adams and colleagues defined sustainability-oriented innovation as making intentional changes to an organization’s philosophy and values, as well as to its products, processes and practices, in order to create environmental and social value alongside economic value. From a systematic review of 127 sources, they derived three contexts through which such innovation progresses (Adams et al., 2016):
This progression is especially useful for public sector organizations and universities. Digitizing a service so that beneficiaries no longer need to travel or use paper is an operational optimization with a direct environmental effect. Redesigning a service around the principle of sharing resources rather than owning them is an organizational transformation. Tackling an environmental problem in the region in partnership with other bodies is systems building, the level that connects with open innovation. The Kingdom has made environmental sustainability and essential needs one of its four national research, development and innovation priorities (Saudi Press Agency, 2022 [Arabic]), so environmental impact has become a question put to initiatives from the outset, where it was once left until after implementation.
Responsible innovation is good innovation management with a wider circle of concern: those who use the solution, those who do not, and those affected by it without ever being asked.
Add these questions to the gates of your organization’s innovation system, so that the team answers them at each stage as far as available knowledge allows. Their value lies in the discussion they open, more than in completing a formal compliance checklist.
| Stage | Responsibility questions |
|---|---|
| Understanding the problem | Who lives the problem? Which groups are least represented in it? Have we heard from them directly? |
| Developing the solution | Who might be harmed if the solution succeeds? Which proxy measures are we relying on, and what do they actually represent? |
| Experiment | Do participants know they are in an experiment? Can they withdraw? What are the stopping conditions? Have the burdens been distributed fairly? |
| Data and artificial intelligence | Are we complying with the Personal Data Protection Law? Who is missing from the data? Can the decision be explained? Who is accountable for it? |
| Environmental impact | What is the solution’s effect on energy and material consumption and waste? Could it create environmental value alongside its core value? |
| Scaling | Did we monitor unintended consequences during the experiment? Do we still have the capacity to adjust after scaling? |
| Ownership and fairness | Does the idea holder know their rights? Have they been given credit? Are the rules public to all parties? |