January 22, 2020, by Communications
Changing how research output is evaluated
In this blog, I introduce a global shift towards the responsible use of publication metrics. It describes the San Francisco Declaration on Research Assessment (DORA), explains what this means for UNM, and also opens a call for academic staff to get involved in that decision making process by joining the UNM Task and Finish group which will be active over the next 6 months.
The outputs from the research are many and varied, including research articles reporting new knowledge, data, and software; intellectual property; and highly trained young researchers. Institutions that employ researchers, researchers themselves, and funding agencies, all have a desire and need to assess the quality and impact of research outputs. It is thus imperative that research output is measured accurately and evaluated wisely.
Individual metrics such as journal impact factor, ABDC Journal Quality List, h-index, citation counts all have deficiencies as tools for research assessment when they are used in isolation.
The signatories of the San Francisco Declaration on Research Assessment (DORA) support the responsible use of publication metrics. This month, all three campuses are signed up to DORA and so this commits the University of Nottingham to promote a real change in research assessment.
What does DORA mean for University of Nottingham?
- Avoid relying on journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.
- Being explicit about the criteria used to reach decisions on hiring, tenure, and promotion, clearly highlighting, especially for early-career researchers, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
- For the purposes of research assessment, considering the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.
What needs to change in our local practices in Malaysia?
We are not absolutely certain at this point. On each campus, we’re establishing a local group to evaluate current practices through the lens of DORA and to develop and oversee the implementation of an action plan, guided by the UNUK Working Group. The purpose of the UNM Task and Finish group is as follows:
- To raise awareness of DORA and its recommendations.
- To make recommendations to UNM Management Board to ensure that responsible use of indicators is reflected in relevant University processes and activities. Some of the processes and activities identified by the UNUK Working Group for review include academic recruitment and promotion, academic ADC and performance monitoring, professional development training for staff and PGRs, and reporting on strategic targets and progress. Amendments to a range of institutional processes are anticipated in order to comply with DORA.
- To establish a process for the ongoing embedding and tracking of DORA’s recommendations.
- To continue conversations with UNUK and UNM about a tri-campus approach.
UNM’s Task and Finish Group on responsible publication metrics
Vacant roles are in red, and so if you are interested in contributing to the decision making at the same time as gaining some experience and insight on university management, please contact Ms Bavani Devarajan by Friday 14 February to express your interest. Task and Finish Meetings will be scheduled on Wednesday afternoons as far as possible.
Chair
Deborah Hall, VP (Research and Knowledge Exchange)
Academic representatives
FASS: Law Chung Lim, Dean
FOSE: Andy Chan, Dean
FASS: Seow Hsin Vonn (Yvonne), ADRKE
FASS/FOSE: Early career researcher (Assistant Prof) tbc
FASS/FOSE: Mid-career researcher (Associate Prof) tbc
Professional Services representatives
RKE hub: Nurul Huda ab Rahman, Research Funding Executive
Professional development: Sutharthan Mariyappan, Head of Professional Development
Graduate School: Deepa Kumari, Head of Graduate School
Human Resources: Manager level tbc
Working group administrative support
Provost office: Bavani Deverajan
-
Post a comment
