Open Science in Teaching

To continue PSI’s tradition of providing high-quality psychology education, we suggest it mandatory that Psychology students at PSI at all levels (i.e., the Bachelor-, Master-, and the Ph.D.-level) are familiar with practices designed to improve the credibility and transparency of knowledge generation and can apply these in their research projects, when appropriate.

Open Science topics are currently not systematically covered in teaching at PSI. We therefore conducted an in-depth evaluation of Open Science teaching needs across the various programs PSI offers. Across programs, we encourage PSI to approach the aims of Open Science through focused and diverse tailor-made practices and by fostering critical reflection on how Open Science practices can be adapted to optimise the potential importance of the research findings.

We make the following main recommendations for improving the teaching of Open Science of PSI:

1. Implement Open Science aims in the curriculum to ensure that all students are informed about Open Science.
2. Implement “doing” Open Science in the various study programs to equip all students with the tools to engage in good research science practices tailored to their method and epistemological approach.
3. Create Incentives for following Open Science aims or practices to acknowledge researchers’ and students’ time and energy investments.
4. Support teaching of open dissemination of research to make research results accessible to everyone and encourage user participation in the research process.
5. Increase the visibility of Open Science as a goal at the department level to demonstrate PSI’s engagement with Open Science in research and teaching, and facilitate the implementation of a culture of trustworthy, transparent, and accessible research culture at PSI.
    

Detailed recommendations for Open Science teaching at PSI

Building on the information obtained and recommendations discussed with members at PSI (the approach is described at the end of this document), we put forward the following recommendations, which i) serve to develop the next generation of academics and psychologists with state-of-the-art research skills, ii) to improve science, and iii) to increase the visibility and outreach of researchers and students at PSI.

1. Implementation of Open Science aims and practices in the curriculum

As a first step, we recommend implementing Open Science practices in the curriculum so that teaching this becomes independent of lecturers’ interests in the topic. Specifically, we recommend to

1. Cover an introduction to the history of Open Science and present both shared and diverging ideas on good research practices independent of special subdisciplines within psychology. We see good places for such an introduction in SVEXFAC03 (examen facultatum, samfunnsvitenskapelig variant) or PSY1050 (Psykologi: Teori og praksis. At a minimum, the following topics should be covered (2x45min):
   • • Introduction to Open science aims, and the challenge of social and economic injustice in access to science and data, user involvement and open access publishing.
     • Introduction to the replication crisis and questionable research practices (including p-hacking, harking, etc.)
     • Relate good research practices in quantitative and qualitative research to the Open Science concept, and present dilemmas and critical reflections.
2. Expand on the importance of critical thinking and how to identify good and questionable research practices in peer-reviewed papers (at least 1x45min). We recommend introducing this topic in PSY1050 (Psykologi: Teori og praksis) because this course aims at familiarizing students with research papers and discussing evidence-based practices.
3. Across subdisciplines, introduce the issue of non-replicability and its implications in one of the first introduction sessions (1x45min), critically discuss which „classical studies“ replicate and which do not, and emphasize topics that replicate. Discussions of why the studies do not replicate should also include a focus on changing historical contexts, the use of WEIRD populations (i.e., the over-sampling of American college students), and cultural differences.
4. Introduce the remedies to overcome questionable research practices in PSY1010 (Innføring i metode) and PSY2014 (Kvantitativ metode). This introduction should answer the following questions: What are the difference between inductive and deductive testing? What is a priori power analysis and why do we need it? What is a preregistration of hypotheses and how and when can this be a helpful tool? How do replication studies within the relevant fields help in advancing science? Starting in 2024, the PSY1010 /PSYC1100 course will include an additional lecture, half of which will be dedicated to exploring the above-mentioned remedies as suggested above. This topic will also be covered in the curriculum.
5. Relate quality criteria for qualitative research, in particular transparency and reflexivity, closer to the concept of Open Science in PSY2013 (Kvalitativ metode; 2x45min) and show how and which elements of Open Science are relevant for qualitative research and depending on the epistemological stance of the researcher.
6. Introduce “refreshers” at the Master’s level. Such refreshers are particularly important for students who did not complete their Bachelor’s at UiO, so that everyone starts with the same knowledge base with regard to Open Science. Such refreshers can be online tutorials on Canvas in the method courses, for instance with the title “What you should know about psychological methods and Open Science from BA-level” (e-learning; 2x45min).

2. Implementation of “Doing” Open Science in the various study programs

With few exceptions (e.g., the elective course PSY2105 [Hands-on science in psychology: Evaluating publications and conducting a research project]; PSY4410 [Applied Qualitative Methods]), students across programs are rarely systematically introduced to doing research and hence to using their knowledge on Open Science practices with regard to quantitative methods, before starting to work on their Master’s thesis. To equip students not only with theoretical but also practical knowledge, we recommend to:

1. Make PSY2105 mandatory for all Bachelor students. When doing so, we deem it important to provide a balanced account that also shows how and which Open Science practices apply to qualitative research.
2. Encourage Bachelor students who want to write a quantitative thesis, to write an empirical Bachelor thesis that intends to replicate a published study. The current practice is that the majority of Bachelor’s students write a theoretical Bachelor thesis. One of the reasons for this decision is the limited time available to conduct a research project. A replication study that works with existing material would overcome this time constraint. This is particularly the case when Bachelor theses are written as part of a larger replication project, such as the Collaborative Replications and Education Project (https://osf.io/wfc6u/). With this recommendation, we do not intend to discourage students from writing a theoretical thesis, in general, but to provide them with a feasible alternative if they are interested in writing an empirical thesis.
3. Restructure the methods education in the Master’s programs to include the potential of Open Science elements. In the current Master’s programs, the students need to choose between qualitative or quantitative methods. We suggest that both qualitative and quantitative methods become mandatory and that Open Science aims and suitable practices are discussed in both of them. We wish to engage the students with both advanced qualitative and quantitative methods to provide the students with a broader perspective to properly prepare them for writing their empirical Master’s thesis. As a result of this recommendation, future Master’s students would receive 10 ECTS in methods, 5 ECTS in qualitative and 5 ECTS in quantitative methods.      

Due to the differences in the various Master’s programs, alternatives could be to

c.1) Restructure the project development courses (PSY4401B and PSY4521) to include an empirical element (similar to a pilot study or pilot interviews for their thesis topic). In addition, we recommend that students can choose between writing a project description/research proposal and writing a preregistration (with the same page constraints).

c.2) Implement a 10 ECTS elective course in the first Master’s term. 

4. Explicitly cover discourses on Open Science in PSY4410 (Applied Qualitative Methods) with 2x45min. The topic should be covered in such a way that the relevance of Open Science practices considering the various epistemological approaches is discussed, and the students are encouraged to participate in critical discussions: What does this mean for qualitative research and depending on the qualitative approach the students adopt? What can be „translated“? What is of less/no importance in qualitative research? Why are transparency and reflexivity (already) important in qualitative research? The students should know about different discourses of Open Science, the different articulations of its aims and relevant practices, and the dilemmas when it comes to embracing and cultivating diversity in epistemologies and knowledge projects.    

5. Introduce a mandatory Open Science course for all Ph.D. students (2-3 ECTS). Excellent groundwork for such a course is provided by the course PSY9311 (Peer reviewing and open science best practices: case affective neuroscience), but the scope would need to be expanded to make it relevant across all subdisciplines in psychology. The Ph.D. course would need to explain different discourses on Open Science, so that it fits every Ph.D. project at the SV faculty, helping to answer the question of how Open Science and which Open Science practices are relevant for the respective Ph.D. project. The scope should clearly expand beyond the topics covered at the Bachelor’s and Master’s levels. As a starting point, we suggest three lectures: 1) Background and discourses on Open Science, 2) The replication crisis and (advanced) ways to handle Open Science in quantitative research (for instance: How can open lab notebooks help? How can I generate reproducible workflows? What established tools can be used? How can I write a registered report? How can I preregister systematic reviews and meta-analyses?), 3) Handling Open Science in qualitative research (e.g., How can Open Science practices be tailored to the researcher’s epistemology and approach? What are ethical dilemmas with regard to data sharing and how to deal with these?). We believe that offering such a course would greatly benefit the Ph.D. education PSI and SV offer and would enable us to be at the forefront of visibly handling Open Science as an institution.

As a mid-term solution until the Ph.D. course is set up, we recommend setting up an agreement with the University of Oslo Library so that students interested in learning more about Open Science can participate in the existing courses offered there and get ECTS from PSI for participating in these courses.

Once the suggested measures at the Bachelor’s and Master’s levels have been implemented, this course might be developed into an elective course so that primarily those who studied outside UiO and have not yet had sufficient first-hand experience with Open Science aims and principles or those who would like to refresh their knowledge can choose this course.

6. Offer an elective course on pedagogies in Open Science teaching at the faculty level and as part of the basic pedagogics competency training. It is important that this course aims at Open Science pedagogics for a variation of research approaches.

3. Incentives for engaging with Open Science aims and practices

Besides their direct involvement in teaching Open Science practices, scholars have a central role modelling function, which means that their own behaviours motivate others (be it their students or colleagues) to behave in a similar way, including taking a critical stance and investigating dilemmas of Open Science. Fostering Open Science practices, including critical engagement, at PSI, can, over time, support the development of a reflexive Open Science culture. To achieve this aim, it is important to acknowledge that Open Science practices take more time (e.g., cleaning and annotating code, writing preregistrations). We therefore recommend to:

1. Offer yearly Awards for students writing a Bachelor’s or Master‘s thesis that addresses Open Science in a particularly good way, such as focusing on participant involvement in deciding the research topic and analysis, employing reflexivity in qualitative research, discussing dilemmas related to Open Science practices, conducting a Registered Report, etc.
2. Support lecturers and researchers in covering and doing Open Science. An example of such support can be to provide additional hours in the timeregnskap for supervising students who follow Open Science practices (such as for going through preregistration documents etc.).

4. Support open dissemination of Research

As elaborated on before, Open Science also includes making research findings accessible and understandable to a broader non-academic audience. To foster this aim, we recommend to:

1. Include dissemination-oriented assignments in all Master’s programs. Such assignments could, for instance, be writing Harvard Business Review-style articles (as currently implemented in PSY4420), summarizing research as infographics, or writing blog posts or podcasts (as currently implemented in PSY4314). Doing so would also directly serve the goal of increasing the practical relevance within the various Master’s programs as recently highlighted as a need from the evaluation committee.
2. Expand the research communication course in the Cognitive Neuroscience Master (PSY4314, 5ECTS) across all study programs as an elective course.

5. Increase the visibility of Open Science as a goal

“Do something good and talk about it.” In light of this statement, our last recommendation concern the visibility of PSI’s emphasis on Open Science aims, practices and dilemmas. Specifically, we recommend to

1. Publish a mission statement for PSI that includes following Open Science aims (trustworthy, transparent, and accessible research) to develop and signal a common understanding for what we stand for as a department. This statement would have to represent the diversity in approaches to Open Science, and engage critically with the movement, including giving attention to the dilemmas it may represent.