What do the users of research want?

The relationship between quality and relevance is a recurring theme in science policy, and the relationship is expressed today especially through terms excellence and impact, which are used in their English form even in Norway. Researchers often want some form of high quality research in a scholarly sense, characterized primarily as original contributions to a research area through publications in prestigious scientific journals. Many organizations and sectors outside of the research system are required to or aspire to use research. These actual or potential users want research to generate some sort of impact – an intended effect in the organization, the sector or society.

But do users and researchers have different perceptions of what makes research good and useful? To study how users view the research system and their own needs for research-based knowledge, we have carried out a large-scale survey with responses from almost 1,900 employees in ministries, directorates and other government agencies in Norway.

Prestige is important to users

We asked the employees about which characteristics of the research and the research environments that they saw as important for the relevance and applicability of the research. The results are shown in Figure 1.

The answers show that the users’ perceptions in many ways correspond to common quality criteria in research. Users are the most concerned with the researchers/research environments being reputable, but they also place great emphasis on publication in recognized journals and that the research is novel. The latter does not necessarily express that what is novel to a user will have scientific originality, but at least there seems to be no fundamental contradiction.

Only a minority of users emphasize previous knowledge of the research environment, that they have commissioned the research themselves, or that the results are published in Norwegian. At the same time, the users are concerned with the fact that the researchers should think about the application of the results: “That it (the research) clearly states what works”, and “that clear implications for policy formulation are described” are considered equally important as originality and even more important than publication in reputable journals.

Positive view of the research communities

Another set of questions was aimed at getting users’ assessments of the research in the users’ specific areas of work. The results are summarized in Figure 2.

The state-employed users largely agree that the researchers and the research communities are perceived as independent, that the communities are concerned with working with relevant issues, and that the research is of high quality. On the other hand, one third of the respondents also fully or partially agree that the research is characterized by a high degree of uncertainty, that it is inaccessible, and that the field has few competent academic environments/researchers.

There are several significant differences between user organizations. For example, respondents from the Ministry of Climate and Environment and the Ministry of Petroleum and Energy in particular expressed that their fields have competent research environments. At the opposite end of this scale we find the Ministry of Local Government and Modernisation, the Ministry of Culture, and the Directorate for Education and Training.

Do users who hold a doctoral degree emphasize other quality criteria?

Between five and ten percent of the respondents hold a doctoral degree and thus have experience from research and scientific publishing. However, their answers to the two sets of questions are very similar to the answers of the respondents without a doctoral degree, see Figure 3 and Figure 4. It can be added that many of the other users in the data material hold a master’s degree.

For the questions about important criteria in research, the respondents with a doctoral degree are less concerned with empirical data and language being Norwegian and that the implications are clearly presented, and they are significantly more concerned with novelty and publishing in recognized journals. Beside this, the differences are insignificant. Regarding the questions about research in their own field, the respondents with a doctoral degree are somewhat more positive about the independence and quality of the research communities, their preoccupation with and media visibility on relevant issues, and they to a lesser extent think that the research is uncertain.

In sum, the answers to these questions express that users in the public sector in many ways put emphasis on the same characteristics of research that the research communities themselves prioritize: originality, regular academic publication, independence, and in many cases an international orientation. From the researchers’ point of view, the most difficult aspect is probably the expectation that they should give clear recommendations for policy development and explain “what works”. In many situations, research might have unclear or several different implications, and it is also not certain that researchers always know specifically how policymaking takes place.

The survey also shows that there is essentially a trusting relationship between researchers and users in Norway, and – at least as a hypothesis – there might be less skepticism about the independence, relevance and quality of research than in other countries.

This blog was originally published as an article in Norwegian in the magazine Forskningpolitikk. You can access the full online version of it here. The survey is part of the work of the OSIRIS center, which studies the societal effects of research. A questionnaire was administered to approximately 6000 government employees in 2019 and received responses from around 1900 people (28 percent). The survey contains information on several topics related to how government employees acquire, disseminate and use research-based knowledge in their work.