Af Lea Laursen Pasgaard, AAU Kommunikation and Public Affairs. Translated by LeeAnn Iovanni, AAU Communication and Public Affairs

Aalborg University (AAU) is the first Danish university to develop a new research indicator that will replace the controversial Danish Bibliometric Research Indicator abolished by the Ministry of Higher Education and Science in December 2021. The new research indicator is thus the formula for the complicated calculation for distributing the basic research funds among the individual departments and faculties that AAU receives each year.

- The new AAU Research Indicator retains the good elements of the previous Danish Bibliometric Research Indicator, but the new model is also forward-looking and fits into a recognizable international framework for research evaluation and advancement where 'Open Science' is a key element along with the standard indicators, Rector Per Michael Johansen says.

A committee consisting of representatives from AAU's four faculties, the Finance and Accounts Department, the VBN Team and AAU Innovation is behind the new AAU Research Indicator. It was created at the request of the Executive Management and AAU's Strategic Council for Research and Innovation.

Focus on societal impact

One of the members of the committee is Birger Larsen, Professor in the Department of Communication and Psychology at AAU. He has investigated bibliometric indicators and agrees with the need for a new indicator as a measure of the departments' research production and their distribution of part of the basic research funds.

AAU's new research indicator consists of two parts, weighted 70 and 30 percent, respectively. Part A focuses on scientific publication and is based on bibliometric methods, i.e. publication points based on publication type and citations. Part B looks at research in an open science perspective and focuses on three areas: Collaboration, Visibility and Openness.

- Part A is in many ways reminiscent of what we know from the old Danish Bibliometric Research Indicator system while Part B is new territory for us. In developing the new indicator, we looked extensively at the latest recommendations and manifestos in the world of research evaluation. We tried to build the elements from ARRA into the model, says Birger Larsen, referring to the EU Commission's agreement document 'Agreement on Reforming Research Assessment (ARRA)'.

The basic idea here is that more qualitative assessments, collaboration, openness and societal impact are of great importance to research today.

Individual targets for individual departments

In its report, the committee provides several examples of how 'Collaboration, Visibility and Openness' can be measured. How strong a department is at collaborating is measured, for example, by the number of scientific publications done in collaboration with researchers from other Danish or international universities. 'Visibility' can, for example, be measured by the number of media mentions and activities on social media, and 'Openness' can be about the proportion of a department's scientific publications that are available via Open Access, i.e. published in journals where all content is freely available to the public on an online platform.

The committee's examples are far from exhaustive. The idea is that the individual departments, in collaboration with the deans, set individual goals for how the department's efforts in the areas in part B can be measured and evaluated.

- There is room for differences between departments and faculties. For example, some SSH departments have a very strong media presence as their researchers often speak to the media and participate in the public debate. For a mathematician or physicist, it can be harder to get their research in the media. Therefore, the goals of the departments' efforts within the different areas of Part B must also be allowed to vary, says Birger Larsen.

- Conversely, virtually all the publications from Physics are published as open access. At the Humanities departments, several books and anthologies are written by publishers where the open access proportion is small, he adds.

AAU has advanced the most with the Danish Bibliometric Research Indicator

Although the Danish Bibliometric Research Indicator was both loved and hated in the research environments, AAU is the only Danish university that has managed to advance in terms of the number of journal articles, the number of publications at level 2 and the number of citations during the period the Danish Bibliometric Research Indicator was used. Part of this success is attributed to the high level of implementation of the Danish Bibliometric Research Indicator at AAU.

A scientific study from Aarhus University shows that the Danish Bibliometric Research Indicator was perceived by many as a control and management tool that created stress and a poor working environment. The study showed that the more a department had adopted the Danish Bibliometric Research Indicator, the poorer the working environment and the more stressed the researchers.

- With the Danish Bibliometric Research Indicator, you could go all the way down to the researcher level and see how many points the individual researcher contributed in a year. At my department, I remember that there was a rule about how many points an assistant professor, associate professor and professor should each generate, says Birger Larsen.

However, the AAU Research Indicator will only measure productivity at the department level, and Birger Larsen believes that this will help researchers see the new model as less stressful. This is also in line with the EU recommendations in ARRA in that the indicator should not be used to put pressure on the individual staff member.

Transparency in developing the research indicator has been important to the committee. The academic councils and academic environments provided input on developing the new research indicator, and this autumn the actual implementation of the research indicator begins. The VBN team is offering a presentation on the new indicator to all heads of department, and the team is also happy to assist in concretizing the goals that the departments can set for part B of the research indicator.

On 24 November, AAU will host a symposium on the Copenhagen campus on research evaluation, impact and indicators where ARRA and the AAU Research Indicator are on the agenda. Among the participants are several research foundations that would like dialogue on how to measure open science parameters, which Part B reflects.

Read more about the event here