Abstract

The pace, depth, and width of the ongoing reform in research assessment is astonishing. By September 2023, over 600 European institutions, mostly universities, has signed an agreement – ARRA (Agreement on Reforming Research Assessment) to reform research assessment practices. The signatories have joint forces in the CoARA (Coalition for Advancing Research Assessment), . The reform addresses three different contexts of evaluation; research organizations and research units, individual research proposals in applications for external funding and individual researchers as they apply for positions, promotions, or internal resources. This is a wide scope, and the three contexts needs different reform-solutions. Our focus is on the latter, the way we are assessing researchers in different contexts. We support the Leiden Manifesto that argue to base assessment of individual researchers on qualitative judgement of their portfolio. Researchers must be assessed by recognising their diverse outputs, practices, and activities. Further, when using quantitative evaluation as a method, this must be supported by expert assessment (qualitative). Metrics can be useful because they can challenge biases tendencies in peer review and expert assessment. But metrics tends to be misused. Therefore, we need to challenge all inappropriate use and call for a knowledge-based approach when using metrics in evaluation.

In this session we will present the Norwegian Career Assessment Matrix (NOR-CAM), a toolbox for recognition and rewards. NOR-CAM concentrate on the dimension where individual researchers apply for positions, promotions, or internal or external resources. Our matrix also broadens the perspective by including all activities, not only research. Although the CoARA and NOR-CAM have somewhat different focus, we see that our national and the European reform is part for the same call for change. In Norway there is an ongoing process of implementing institutional versions of NOR-CAM, and this process runs hand in hand with our involvement in the CoARA, joined by almost 20 Norwegian institutions. In our national NOR-CAM-network we investigate the synergies between the European, the national and the institutional processes in reforming our evaluation-system. Hence, the NOR-CAM network has been approved as the Norwegian National Chapter by CoARA.

Recognising that this reform will take time, that it will depend on many actors acting together and that we are able to solve some quey issues, we have high expectations to the working groups set up by CoARA, that they will be arenas for some early collective steps. This is arenas where we can agree on collective action, where we can set up workshops for solving challenges and discuss “routes out of the jungle”. One challenge we tend to return to is one of the most fundamental parts of the academic ecosystem, namely how we identify, recognize and document the more diverse forms of results and competences, in other words: how do we recognize quality when a whole range of activities is taken into account?

Reforming the assessment system is a huge task, involving dimensions along culture, governance, academic fields, infrastructure, information interoperability and many more. We recognise also that no one can do this by themselves. But one thing seems certain; we are about to enter a profound and comprehensive change process that will both involve and affect the whole knowledge system. No institution or academic field will be unaffected by this in the coming years. What is also certain; this will land on research managers desks, both to find solutions and to give advice!

Learning outcomes:
Our ambition is that participants in this session comes out with updated information about the evaluation reform, including some of its dilemmas and challenges. In this way they can be at the forefront of what is coming. We hope the participants can go back home to their own institutions, having signed the ARRA or not, and challenge their old evaluation systems and initiate change towards the lines of CoARA. As a research manager you will be inspired to engage in developing new strategies and tools for assessing researchers when they apply for positions, promotions, and funding. We also hope that the participants will be motivated to engage in different networks, both national and international, that addresses these issues and contribute to those communities by ask questions, participate to find solutions and to share good practices.

Both presenters are part of the Open Science team at Universities Norway and we are central in coordinating NOR-CAM as a National Chapter and CoARA activities in Norway.