Jan Ole Hesselberg

PhD candidate

Norwegian version of this page

Email janohes@student.sv.uio.no

Username

Download business card

The PhD Project

I Norway, several billion Euros are granted through application processes. However, the reviewers often disagree in their scoring of the proposals.

Why are the agreement levels low? And what can be done about it?

That is the topic of my project.

Funding and collaboration

The project is funded by the Dam Foundation (Stiftelsen Dam).

The Norwegian Cancer Society, The Kavli Trust and The Norwegian Research Council contribute with review data from their funding programs.

Academic interests

Judgement and decision making
Peer review
Open science

Background

I am a psychologist with a specialisation in clinical adult psychology.

I am the chief program officer at the Dam Foundation and the program manager of the Kavli Trust Programme on Health Research. I am also a member of The Norwegian National Committees for Research Ethics and I host a podcast called "Psykologlunsj".

Back in the days I worked on two popular science shows on Norwegian TV. "Folkeopplysningen" and "Typisk deg med Petter Schjerven".

For more info, check out my LinkedIn profile.

Research team

Supervisors

Knut Inge Fostervold, Associate Professor, Department of Psychology, University of Oslo
Pål Ulleberg, Associate Professor, Department of Psychology, University of Oslo
Ida Svege, Head of program development, Stiftelsen Dam

Seeber, Marco; Svege, Ida & Hesselberg, Jan Ole (2024). Improving the efficiency of research proposals evaluation: A two-stage procedure. Research Evaluation. ISSN 0958-2029. doi: 10.1093/reseval/rvae020.
Hesselberg, Jan Ole; Dalsbø, Therese Kristine; Strømme, Hilde; Svege, Ida & Fretheim, Atle (2023). Reviewer training for improving grant and journal peer review. Cochrane Database of Systematic Reviews. ISSN 1469-493X. doi: 10.1002/14651858.MR000056.pub2. Show summary
Background Funders and scientific journals use peer review to decide which projects to fund or articles to publish. Reviewer training is an intervention to improve the quality of peer review. However, studies on the effects of such training yield inconsistent results, and there are no up‐to‐date systematic reviews addressing this question. Objectives To evaluate the effect of peer reviewer training on the quality of grant and journal peer review. Search methods We used standard, extensive Cochrane search methods. The latest search date was 27 April 2022. Selection criteria We included randomized controlled trials (RCTs; including cluster‐RCTs) that evaluated peer review with training interventions versus usual processes, no training interventions, or other interventions to improve the quality of peer review. Data collection and analysis We used standard Cochrane methods. Our primary outcomes were 1. completeness of reporting and 2. peer review detection of errors. Our secondary outcomes were 1. bibliometric scores, 2. stakeholders' assessment of peer review quality, 3. inter‐reviewer agreement, 4. process‐centred outcomes, 5. peer reviewer satisfaction, and 6. completion rate and speed of funded projects. We used the first version of the Cochrane risk of bias tool to assess the risk of bias, and we used GRADE to assess the certainty of evidence. Main results We included 10 RCTs with a total of 1213 units of analysis. The unit of analysis was the individual reviewer in seven studies (722 reviewers in total), and the reviewed manuscript in three studies (491 manuscripts in total). In eight RCTs, participants were journal peer reviewers. In two studies, the participants were grant peer reviewers. The training interventions can be broadly divided into dialogue‐based interventions (interactive workshop, face‐to‐face training, mentoring) and one‐way communication (written information, video course, checklist, written feedback). Most studies were small. We found moderate‐certainty evidence that emails reminding peer reviewers to check items of reporting checklists, compared with standard journal practice, have little or no effect on the completeness of reporting, measured as the proportion of items (from 0.00 to 1.00) that were adequately reported (mean difference (MD) 0.02, 95% confidence interval (CI) −0.02 to 0.06; 2 RCTs, 421 manuscripts). There was low‐certainty evidence that reviewer training, compared with standard journal practice, slightly improves peer reviewer ability to detect errors (MD 0.55, 95% CI 0.20 to 0.90; 1 RCT, 418 reviewers). We found low‐certainty evidence that reviewer training, compared with standard journal practice, has little or no effect on stakeholders' assessment of review quality in journal peer review (standardized mean difference (SMD) 0.13 standard deviations (SDs), 95% CI −0.07 to 0.33; 1 RCT, 418 reviewers), or change in stakeholders' assessment of review quality in journal peer review (SMD −0.15 SDs, 95% CI −0.39 to 0.10; 5 RCTs, 258 reviewers). We found very low‐certainty evidence that a video course, compared with no video course, has little or no effect on inter‐reviewer agreement in grant peer review (MD 0.14 points, 95% CI −0.07 to 0.35; 1 RCT, 75 reviewers). There was low‐certainty evidence that structured individual feedback on scoring, compared with general information on scoring, has little or no effect on the change in inter‐reviewer agreement in grant peer review (MD 0.18 points, 95% CI −0.14 to 0.50; 1 RCT, 41 reviewers, low‐certainty evidence).
Hesselberg, Jan Ole; Fostervold, Knut Inge; Ulleberg, Pål & Svege, Ida (2021). Individual versus general structured feedback to improve agreement in grant peer review: a randomized controlled trial. Research Integrity and Peer Review. ISSN 2058-8615. 6(12). doi: 10.1186/s41073-021-00115-5.
Hesselberg, Jan Ole; Dalsbø, Therese Kristine; Strømme, Hilde; Svege, Ida & Fretheim, Atle (2020). Reviewer training for improving grant and journal peer review. (Protokoll). Cochrane Database of Systematic Reviews. ISSN 1469-493X. doi: 10.1002/14651858.MR000056. Full text in Research Archive
Teigen, Karl Halvor; Andersen, Bjørn Sørskot; Alnes, Sigurd Lerkerød & Hesselberg, Jan Ole (2019). Entirely possible overruns: How people think and talk about probabilistic cost estimates. International Journal of Managing Projects in Business. ISSN 1753-8378. 13(2), p. 293–311. doi: 10.1108/IJMPB-06-2018-0114. Full text in Research Archive

View all works in Cristin

Røislien, Jo; Frøslie, Kathrine Frey; Stavseth, Marianne Riksheim; Langaas, Mette; Kvaløy, Jan Terje & Hesselberg, Jan Ole (2024). Nakne Tall (6:9) "Er du helt sikker?". [Internet]. Podkast NettOp UiS. Show summary
Verden er full av usikkerhet, og svaret på enhver statistisk analyse av tall vil derfor også ha usikkerhet bakt inn i seg. Jo Røislien snakker med psykolog og forskningsfinansiør Jan-Ole Hesselberg om pizza og powerposing og hvor galt det kan gå når man fisker rundt i tallene på leting etter noe spennende. Ekspertpanelet forsøker å forklare statistikkens kanskje mest brukte og misbrukte tall: p-verdien.
Røislien, Jo & Hesselberg, Jan Ole (2023). Hvor ofte er aldri? Tidsskrift for Den norske legeforening. ISSN 0029-2001. 143(18), p. 1–3. doi: 10.4045/tidsskr.23.0731. Show summary
Vi bruker ord for å beskrive usikkerhet og sannsynlighet hele tiden. Samtidig er vi ofte svært uenige om hva disse ordene faktisk betyr.
Meyer, Jonas Hansen; Jordal, Preben; Økland, Ingunn; Søbye, Espen; Kvam, Anna Serafima Svendsen & Hesselberg, Jan Ole [Show all 8 contributors for this article] (2022). "Allerede før boken er åpnet" – om (in)habilitet i litteraturkritikken. Prosa - tidsskrift for skribenter. ISSN 0805-276X.

View all works in Cristin

Published Mar. 12, 2020 9:12 AM - Last modified Mar. 11, 2024 11:29 AM