Abstract

A call for more responsible research evaluation is spreading wide and fast. It has become apparent that the consequences, both intended and unintended, of research evaluation cannot be ignored. As a response, several guidelines presenting the ideals of responsible research evaluation have been published, starting from the San Francisco Declaration on Research Assessment in 2012. In July 2022, the European University Association, Science Europe and the European Commission published the Agreement on Reforming Research Assessment, which sets a shared direction for changes in assessment practices for research, researchers and research performing organisations. Research organisations, funders and individual researchers all over the world are signing up to these ideals, but realizing ideals needs good implementation. The SCOPE Framework for responsible research evaluation was developed by the INORMS Research Evaluation Group as a practical tool for practicing research managers and administers. It allows them to assess existing evaluations or to design new ones in line with best practice in research assessment. SCOPE encompasses the entire evaluation system allowing any evaluator to evaluate any entity using any relevant mechanism.
SCOPE is an acronym which stands for S – STARTing with what you value when seeking to e-valu-ate; C- Consider the CONTEXT in which you are evaluating – the who and the why?; O – Explore all OPTIONS for evaluating; P – PROBE your evaluation for unintended consequences and discriminatory effects; and E – EVALUATE your evaluation. SCOPE operates under three principles: 1) Evaluate only where necessary; 2) Evaluate with the evaluated and 3) Utilise evaluation expertise. The framework has been used by a wide range of organisations: funders, universities and publishers, to design responsible evaluations.
This session will situate the SCOPE framework within the range of responsible research assessment ideals, explain the SCOPE process and provide the participants a concrete tool towards conducting more responsible research evaluations in a world where the ideals for responsible research assessment are spreading, but not enough expertise, nor relevant training is yet available.