Abstract

The Impact Planning and Evaluation Network (iPEN) was established in 2014 and spans all seven of New Zealand’s Crown Research Institutes, which collectively employ 4,000 staff. Having become aware of numerous constraints on the delivery of research impact, iPEN undertook surveys and interviews with our scientists and researchers. The questions focused on what it takes to maximise the impact of research. We applied systems thinking and analysis, and tested the findings with a wide range of stakeholders, including research administrators and managers, and funding agencies. The disciplinary basis of the approach drew on evaluation perspectives and systems analysis.

Our findings fell into three key areas:

1. A need to expand understanding of the activities and steps that require funding for science and research to be impactful to encompass relationship development, the full range of impact pathways, and other activities central to research impact. We depict this as an impact creation cycle, as opposed to the linear input-activity-output type of model. This expanded understanding is proving to be a valuable engagement tool with funders and end-users.

2. The identification of seven key enabling themes within the current system that support the delivery of impact from science and research. These include the need for clear system priorities, the central role of relationships, and the importance of funding that is both stable and flexible. These themes are nuanced, and applicable at the national, institutional, portfolio, and individual project levels.

3. An understanding of deeper systemic issues. The components of science and research systems are complex and interconnected, sometimes in unexpected ways that produce unpredictable outcomes. Using system dynamic modelling tools (e.g. causal loop diagrams) and system archetypes, we identified several systemic issues, including: an output and activity-focused KPI trap; a system that wishes to maximize impact but is undermining itself through financial imperatives; and gaps in strategic direction and a lack of signals to guide the science system, which create a drift and dispersal of effort. These and other issues propagate themselves throughout the system.

If we ignore the complexity within and across these three areas, we fall back on addressing individual issues in isolation from the rest of the system. This can lead to treating symptoms, not causes; missing easy wins; and undertaking interventions that will have no or limited effect unless other aspects are accounted for; for example, improved prioritization has limited effect if not reinforced through process change; and training people to better deliver impact has limited effect if there is no career recognition.

To increase impact from science and research institutions, we suggest the areas to focus on are:
• clarifying societal needs and research priorities
• making more space for strong relationships
• decoupling science excellence and impact
• increasing funding stability and flexibility
• increasing reward for and recognition of impact
• reinforcing change through administrative systems and processes (e.g. funding application design, contracting models, KPIs)
• building impact capability and capacity.

For research administrators and managers these insights offer opportunities to create a better environment for researchers to deliver impact.