Assessing research performance fairly

FAU signs the San Francisco Declaration on Research Assessment

Which criteria can be used to evaluate and assess how successful a research achievement is? Until now, funding organisations, academic institutions and other decision-making bodies have often relied upon the Journal Impact Factor (JIF) of the journal where the research was published. Considering that outcomes of research performance assessments can influence funding decisions and academic careers, relying on a single factor oversimplifies a process that includes a far wider range of criteria. FAU is one of the first universities in Germany to support the ‘San Francisco Declaration on Research Assessment’ (DORA), which promotes fair, transparent and quality-based criteria in assessing academic research.

If medals were awarded to athletes based on the reputation of the magazines that publish articles about them, the more prestigious the better, people would begin to ask questions. But this is exactly the situation that researchers are faced with throughout the world: Their performance is mostly assessed on how many articles they have published in prestigious publications that have a high Journal Impact Factor (JIF). Further criteria that assess their individual contribution to the research either play an inferior role or are not considered at all.

Fair assessments require more than one criterion

It is widely accepted that academic research covers a most diverse range of topics and results in specialist publications with new knowledge, data, reagents and software, as well as intellectual property and indirectly also new young researchers who are starting out on their academic careers or receiving training while working on their research projects. Funding organisations, academic institutions and steering committees are themselves faced with the demanding task of assessing the quality of research findings.

The JIF is frequently used as the most important factor to compare research productivity between individual researchers and institutions. The JIF is an indicator of the influence of an academic journal and specifies an average value of how often an article has been cited in other publications. It has recently drawn criticism as an objective measurement of research quality in a specific article because it does not include a weighting factor for assessing the reader and citation potential in a specific field.

DORA intends to establish fair, transparent and quality-oriented criteria to measure research performance. It includes, among others, the following recommendations:

• Assess the quality of research on its own merits and in a discipline-specific manner
• Use open access publication routes
• Disclose the criteria that form the basis for hiring, promotions, and appointments Consider other research achievements beyond publications
• Avoid journal-based metrics such as the JIF in funding, appointment and promotion procedures
• Educate young researchers that the academic content of an article is far more important than the journal it is published in.

FAU has signed the declaration to acknowledge these recommendations. But what is FAU doing to actively ensure fairness in research performance assessment?

In appointment procedures or for research proposals to funding organisations such as the German Research Foundation (DFG), criteria such as the academic excellence of research achievements, acquisition of funding, prizes and awards, involvement in high-ranking professional societies or committees of funding organisations, national and international networking, scientific integrity and equal opportunities, innovation potential in research and teaching, transfer and communication achievements, or even the training of young researchers have long played an important role at FAU.

In signing the DORA declaration, FAU has committed to review and where necessary, to change current assessment practices at all points where research performance is assessed and compared. This is the only way to ensure that research performance is always evaluated fairly and in accordance with transparent criteria.

The DORA declaration was proposed in 2012 at the annual meeting of the American Society for Cell Biology in San Francisco and has already been signed by the European Research Council (ERC), the European University Association (EUA) and the German Research Foundation (DFG), among others. In Germany, the FAU is one of the first universities to sign the declaration.

Further information

DORA https://sfdora.org/

DORA at FAU

Dr. Christian Schmitt-Engel
support for young academics
christian.schmitt-engel@fau.de
Phone: +49 9131 85-20722

Bibliometrics and scientometrics

Dr. Joachim Hennecke
FAU University Library
joachim.hennecke@fau.de
Phone: +49 9131 85 22167

Open Access and Open Science

Markus Putnings
FAU University Library
markus.putnings@fau.de
Phone: +49 9131 85 27835 or 24797