Policy Tools: Lessons from Missions in Practice
By Anna Goulden
This is the second blog in the series on policy tools, the first of which can be found here.
Tools and Toolkits
Through our analysis of policy tools and toolkits, it has become evident that the meanings associated with the term “tool”, in the context of mission-oriented policies and policy more generally, are nuanced, subjective and varied. These variations and nuances in both interpretations and usages of the term have been central to our research on the topic. In particular, we have been exploring the differences — and similarities — between policy “toolkits” and a policy “tools.”
While our understanding of the two terms is still evolving, a key distinction has emerged in terms of their function: while a policy toolkit tends to curate, organise and present a set of tools, resources or guidance (plural) for practitioners to draw from, a policy tool is an instrument (singular) designed for application in an, often more specific, policy context or challenge. In the context of mission-oriented innovation, an example of the former (policy toolkits) would be the OECD’s Mission-Oriented Innovation polices online toolkit or Fraunhofer’s Mission-Oriented Innovation Policy for Transformative Change toolbox. On the other hand, examples of the latter (policy tools) could include this IIPP framework for assessing value creation, developed as part of the IIPP-BBC project and scoping report Creating and Measuring Dynamic Public Value at the BBC (see figure on page 10), or non-written/visual formats such as UNDP’s systems transformation portfolio approach and the London Borough of Camden’s mission-oriented community wealth fund.
In this blog series, I share our analysis of both policy tools and toolkits, and try to reflect how our understanding of both terms has developed throughout the project. In particular, our engagement with practitioners in IIPP’s Mission-Oriented Innovation Network (MOIN) has shed insights on the policy and organisational contexts in which such tools and toolkits are applied and therefore the need for a contextualised understanding of both terms. Nevertheless, regardless of how we may define policy toolkits relative to policy tools, a uniting and distinguishing feature of both categories is that they are designed with practical application at the centre. In the context of policy areas and approaches like mission-oriented innovation, both tools and toolkits can be a means of bridging the gap between theory and practice by providing practitioners with tangible resources to support them in their work. In a sense, this is the core of our interest in the topic and the subject of exploration in this blog series.
Practitioner Perspectives: Contexts, Current Practice and Future Needs
Contextualising tools in current practice and needs is critical and, therefore, understanding practitioner perspectives has been an important pillar of our tool analysis to date. This analysis is based on ongoing engagement with practitioners in the MOIN Network, via survey questions, discussions and interviews, as well as secondary data on mission-oriented innovation practitioners more broadly, in particular the survey by the Organisation for Economic Co-operation and Development (OECD) and the Danish Design Centre (DDC) on ‘Mission-Oriented Innovation Needs’. Although the analysis has centred on the application of policy tools in mission-oriented policy practice, there are transferable learnings for policy tools more generally. It has focussed on three main areas of enquiry, each of which will be explored in this blog:
- Contexts: What are the operational contexts and environments in which ‘mission practitioners’ (those practicing a mission-oriented approach) are or would be using tools?
- Current Practice: Which tools are mission practitioners already using in their work and how?
- Needs: What do mission practitioners need in terms of tools, now and in the future?
1. CONTEXTS
Research observation: a big proportion of mission practitioners identified as being in the “implementation” phase.
In a survey circulated with IIPP’s Mission-Oriented Innovation Network (MOIN) of public sector practitioners in May 2023, 43% of respondents identified themselves as being in the “implementation” phase of a mission (Figure 1 below). An even greater proportion, 65%, of respondents — in a 2022 OECD and DDC survey on mission-oriented innovation needs — self-defined as being engaged with implementing missions. This could mean developing mission portfolios, implementing initiatives, or anything else associated with putting missions into practice.
Research observation: in conversations about tools, practitioners emphasised the importance of organisational and contextual factors.
The centrality of the organisation in thinking about or using tools emerged clearly in my discussions with practitioners. This is exemplified by a quote from one practitioner who, when asked about the sorts of tools he uses, responded “our main tool is the organisation itself.”
More broadly, contextual factors were identified as instrumental to tool usage and practitioners highlighted the need for tools to adapt and evolve as contexts change.
“One thing we’ve learned in using MOI tools at [my organisation] is nothing is ever stable/replicable. It’s a constant process of adaptation/iteration across missions in our portfolio and for individual missions across their journey.”
“Our main tool is the organisation itself.”
Research observation: tools are often a means of responding to a specific challenge or question.
In my discussions with practitioners, I asked them to tell me about situations in which they looked to tools. In most of their responses, it was apparent that the starting point was a question or challenge — and therefore that the impetus to find a tool was closely tied with the need for a resolution. “We go to tools with specific needs or questions in mind,” said one interviewee. Another raised the fact that they often turn to tools when scoping a new project, problem or approach to understand best practice or potential strategies.
“We go to tools with specific needs or questions in mind.”
2. CURRENT PRACTICE
Research observation: best practice, methods and tools for mission-oriented policies are still emergent.
Since mission-oriented innovation is still an emergent field, many of the tools in the space are also still under development. “Working with mission-oriented innovation is a new and emerging field which still lacks good practices, tools and methodologies,” writes the DDC/OECD in their Mission-Oriented Innovation Needs Assessment Survey.
From my discussions so far, it seems that the topic of tools is one on which those practicing missions are keen to engage and therefore I have sensed a demand for — or at least an interest in — tools that can support practitioners on their mission journeys.
In a 2022 MOIN Member Survey, out of nearly twenty potential areas of assistance identified by IIPP, the following five (in order of popularity) received the most votes: monitoring / evaluation / impact assessment; methods and tools; legitimacy and consensus building with stakeholders; mobilising the ecosystem; and collaboration across silos (Figure 2 below). This demand for tools is also reflected in data from the previously mentioned OECD/DDC survey in which “skills, methods and capacity” was selected by practitioners as the most common area requiring external help (selected by 67.5% of respondents).
Research observation: there is diversity in the types of tools practitioners are using, which extends beyond digital/physical forms.
It would be wrong to assume that this perceived lack of tried-and-tested methods and best practice means mission practitioners are not using tools to inform their work. In fact, my engagement with practitioners has revealed the breath of sources and materials being developed and applied (Figure 4 below).
This has been particularly evident in the Tools Learning Series that IIPP is convening with its Mission-Oriented Innovation Network: a space of learning and exchange among the MOIN community on tools and toolkits for mission-oriented policies developed in the field. In the first workshop of the series, participants emphasised that their tool usage extends beyond physical or digital materials. As visible in our poll responses (Figure 3 below), the “tools” MOIN members are using includes theories (e.g. feminism), communities (e.g. communities of practice, calls), and processes (e.g. governance and participatory processes). As mentioned in the introduction to this blog, insights like these from practitioners have helped us to broaden the remit of our analysis to extend from mission-oriented policy “toolkits” to “tools” in their many forms.
Research observation: many practitioners are developing their own tools in order to reflect on, share and codify practice.
It also became clear to me that practitioners are not only taking tools from elsewhere but also creating their own. “We often make tools to codify what we’ve already done,” said one practitioner I spoke to. This process of developing, adapting and sharing tools was perceived as a way of reflecting on, codifying, iterating and exchanging practice — both within one’s own organisation and beyond. This is exemplified by a quote from one design agency official who said, “[toolkits are] a way to reflect on what we’re doing in action.”
The tools that practitioners I interviewed are developing included: theories of change; mission maps; design tools such as prototyping and canvassing; governance models; digital open-source software; participation tools; stakeholder maps; and data management and sharing tools.
“[Toolkits are] a way to reflect on what we’re doing in action.”
3. NEEDS
Research observation: the challenges practitioners are facing vary significantly with context but there are common threads.
In the first section, on contexts, I described how often practitioner tools usage is highly challenge-driven and contextually-dependant. Since challenges tend to be closely tied to organisational and contextual factors such as relationships with stakeholders, buy-in from senior management, organisational structures, or the external innovation landscape, they can vary greatly in nature. Plus, the scope and nature of each mission varies too: in the 2022 MOIN Survey, priority projects being worked on by MOIN members appeared to vary significantly (Figure 5 below) and health, social, environmental and technological challenge types were selected almost equally in the DDC/OECD Survey.
On face value, this could pose challenges for the development and implementation of tools which may rely on a degree of generalisability between use cases. Yet, from my research, some commonalities between challenges mission practitioners are facing did emerge: I’ve called these “challenge clusters” — challenges which are not identical due to contextual variations but share core features. These included:
The OECD/DDC survey also identified some key challenge groups (Figure 7 below) such as: financial challenges including lack of targeted resources; structural challenges like budgeting timelines; political challenges such as electoral cycles; methodological challenges including a lack of evaluation tools; challenges associated with the mission roadmap like data collection; and, lastly, the risk of failing due to silos and lapses of momentum or support.
The survey also solicited feedback “methodological challenges” in which a lack of a concrete governance framework was identified as the most impactful challenge, followed by a lack of policy learning platforms/networks and evaluation tools. Respondents were also asked about the areas in which they need “external help”: this solicited a range of responses, with “skills, methods and capacity” being the most common. However, not all of these responses were consistent with the challenges identified in Figure 7, perhaps suggesting that some challenges are perceived as better suited to external help than others.
Research observation: practitioners expressed the need for cases stories of missions in action.
The Toolkit Typology I developed, outlined in more detail in the first blog in this series, identifies four main functions by which toolkits tend to add value (Figure 2 in above link). These are: collation (of resources into one, central location), systematisation (of theory, knowledge and tools into a process or component parts), exemplification (of cases or approaches) and education (in necessary skills, knowledge and techniques). Using this typology as a framework, I asked practitioners what sort of tools they found to be effective: “casebooks” were selected more frequently of all the typologies, with a fairly even spread across the board.
The demand for cases and examples of mission-oriented innovation policy was also apparent in my discussions with practitioners: four out of four of the people I spoke to at the 2023 Mission-Oriented Innovation Network Gathering identified case studies/stories as important tools. They explained the ways in which cases can be useful: from providing inspiration, to demonstrating best practice when scoping a project, or informing potential responses to a problem or barrier.
“[Toolkits are] a way of codifying [practice] and making it transparent and shareable for others.”
Research observation: clarity and context were identified as key to a tool’s utility.
Yet several people I spoke to noted that contextualising and applying these cases to a given challenge, project or context can be difficult — and, with one interviewee, we went on to discuss why framing is key.
“Case studies often lack context and detail on the ‘why.’”
It seemed that, for practitioners, much of the value of tools and toolkits is in their ability to present information with clarity and concision. They can help busy practitioners to respond to often urgent challenges, quickly — rather than spend time rummaging through long books or reports.
“We don’t have time to read long reports.”
The Future of Mission-Oriented Policy Tools
So, what might these insights from practitioners mean for the development and application of policy tools and toolkits? In the next blog of this series, I will share some reflections on how tools and toolkits can respond to some of the complex, changing needs of current mission practice.
