institutions and research groups
1. [NAME OF INSTITUTION]
SECRETARIAT: A short introduction on establishment and development of the institution and its organization.
SECRETARIAT: Fact sheet
1.1 Evaluation of [Institution]
1.1.1 Organisation, leadership and strategy
ASSESSMENT: overall reasoning
• How do you review the leadership of the research area on an institutional level?
• Does the institution have adequate goals and a suitable or an unsuitable strategy to reach them?
• How do you consider the institutions’ strategic focus (or lack there of), taken into account its publication strategies, the national and international research collaboration?
• How does the institute make use of external research funding? Are the use of this funding reasonable, and/or is there room for improvements?
1.1.2 Institutional following up on previous evaluations
ASSESSMENT: overall reasoning
• Consider conclusions and recommendations from previous evaluations, and give your opinion on the way the reviews have been followed up.
1.1.3 Research environment (if relevant)
ASSESSMENT: overall reasoning
• How do you review the institutions policy for maintaining a fruitful environment for
production and exchange of knowledge? (i.e. seminars, summer schools, guest lectures and scholars, etc)
1.1.4 Resources and infrastructure
ASSESSMENT: overall reasoning
• Does the institution provide adequate resources and infrastructure?
• Does the research area make good use of these?
1.1.4 Research personnel
ASSESSMENT: overall reasoning
• Are the area’s hiring and career development practices consistent with best practice?
• Are PhD candidates, post-docs and junior faculty adequately trained and mentored?
• Has the institution implemented the European Charter and code and been awarded the brand “HR Excellence in Research, or what are the plans to implement the Charter?
• Is there a transparent career path?
• Is there sufficient national and international mobility of researchers?
173
• Is the balance among the research personnel appropriate in terms of gender, age and diversity?
1.1.5 Research production and scientific quality
ASSESSMENT: reasoning
Numerical scale, scientific quality, 5-1 (excellent–weak)
• To which extent does the institution pursue policies to improve and facilitate scientific performance of high quality?
• How is the productivity, the degree of originality and international profile?
• Evaluation of the cases from the institutions in the research area
• Has the institution contributed to advancing the state of the art in the research area /scientific discipline/ to interdisciplinary production of knowledge?
• How does the institution make use of interdisciplinary approaches, when these are relevant?
1.1.6 Interplay research-education: impact on teaching
ASSESSMENT: overall reasoning
• How is the balance between teaching and research?
• Are there established linkages between the research and the study programmes offered by the institution?
• Does the institution have a focus /strategy to secure / improve the interplay of teaching and research?
• How are eventual challenges addressed and handled?
• To what extent are students involved in staff research?
1.1.7 Societal relevance and impact
ASSESSMENT: reasoning + identify best cases
• Does the institution have strategies for dissemination, user-involvement and knowledge exchange? How do you review the strategies?
• Does the institution document relevant dissemination/knowledge exchange activities?
• Does the ongoing research at the institution have a linkage/association to thematic priorities set out in the Norwegian Government’s Long-Term Plan for Research and Higher Education and other relevant policy documents?
• To what extent does research in the area benefit the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia? What is your overall view?
1.1.8 An overall review on profile, scientific quality and impact on institutional level
ASSESSMENT: overall reasoning
1.1.9 Feedback
174
1.2 Evaluation of [Research group A]
Short description of the research group.
ASSESSMENT: overall score 5-1
1.2.1 Organisation, leadership and strategies
ASSESSMENT: overall reasoning
• How do you review the leadership of the research group?
• How do you view the group’s intellectual focus and strategy to reach them? Please take into account its publication strategies, the national and international research collaboration.
• Does the group make use of external research funding, and eventually how? Are the use of the external funding reasonable, and/or is there room for improvements?
• Does the research group contribute to the institution’s overall goals or not?
• To which extent does the institution pursue policies to improve and facilitate scientific performance of high quality?
• Does the institution provide adequate resources and infrastructure, and how does the research group make use of them?
1.2.2 Research personnel: including recruitment, training, gender balance and mobility
ASSESSMENT: overall reasoning
• How is the group’s hiring and career development practices? Are they consistent with best practice?
• How to you view the training and mentoring of PhD candidates and post-docs?
• Is the balance among the research personnel appropriate in terms of gender, age and diversity?
• How is the national and international mobility of researchers? Is it sufficient /insufficient and in which way(s)?
1.2.3 Research production and scientific quality
• How is the productivity of the research group, the degree of originality and its international profile?
Has the group contributed to advancing the state of the art in its discipline(s)? If yes, how?
• Does the group make use of interdisciplinary approaches, where these are relevant? How?
• How do you review the quality of the research overall?
1.2.4 Networking
ASSESSMENT: overall reasoning
• Does the group make good use of collaboration, nationally and internationally, to advance its strategy and produce high-quality, relevant research?
1.2.5 Interplay research-education: impact on teaching (if relevant)
ASSESSMENT: overall reasoning
• Does the research group contribute to educational activities?
• To what extent is the research of the group relevant for the study programmes at the host institution or other institutions?
175
1.2.6 Societal relevance and impact (if relevant)
ASSESSMENT: overall reasoning
• Does the research group document relevant dissemination/knowledge exchange activities?
• To what extent does research in done by the research group benefit the economy, society, culture, public policy or services, health, the environment or quality of life, beyond
academia? What is your overall view?
1.2.7 Overall assessment
ASSESSMENT: overall reasoning
• What is the overall profile, and scientific quality of the research group?
• To what extent is the research group linked to / have an impact on the research environment at its institution?
• What is the overall significance of the research group in a national research area context?
1.2.8 Feedback
176
Appendix J: Template for an assessment of the ten most important publications listed by the institutions
Template for an overall assessment of the ten most important publications listed by the institution
The institutions have been invited to submit a list of ten most important publications. These
publications are listed as the attachment of the institutional self-evaluation, and also available as pdf or open access links for further information.
The assessor should provide an overall assessment of these listed publications by the institution. The assessment is overarching, however, the publications can be consulted if/when relevant.
The overall assessment should be provided with the grading scale for scientific quality, along with reasoning.
Note that not all of the questions involve a quality criterion (for instance, to what extent are the publications interdisciplinary or co-authored does not imply a normative judgement), these criteria are proposed to link the assessment of publications to the overall assessment of the institutional aims and strategies for the field.
How would you assess the selected publication outlets (i.e. significance and quality of journals, publishers, book series)?
How would you assess the originality and significance of the publications within its designated field, nationally and/or internationally?
Are the submitted ten publications representative of the discipline in this institution? (i.e. do the
publications represent few/many of the researchers and sub-themes of the discipline in this institution?)
If relevant: To what extent do the publications contribute to interdisciplinary research?
If relevant: To what extent do the publications include co-authoring with significant researchers on the field (nationally and internationally)?
To what extent do the publications reflect the stated thematic, theoretical and/or methodological foci of the institution?
How would you in broad terms assess these ten publications?
177
Appendix K: Template for an assessment of the publications of listed members at the research groups
Template for an overall assessment of the publications of listed members at the research groups
The research groups have been invited to submit one publication per member listed in the
evaluation. The publication could be a scientific article or a book chapter. For monographs and other publications exceeding 30 pages, the main ideas and findings of the publication should be indicated.
The selected chapter(s) should not exceed 50 pages. The panel will consider when external referees are needed for further assessment of the publications.
The assessors should review all the submitted publications in terms of their quality, and provide a single assessment for all of the publications submitted by the research group, following the template underneath. The assessment should be written with the assumption that parts of the text can be used for the section with the title “Research production and scientific quality”.
Please provide an overall assessment using the grading scale for scientific quality (5-1), as well as a short reasoning for the grade.
Note that not all of the questions involve a quality criterion (for instance, to what extent are the publications interdisciplinary or co-authored does not imply a normative judgement), these criteria are proposed to link the assessment of publications to the overall assessment of the research group aims, scope and strategies.
How would you assess the selected publication outlets, i. e. significance and quality of journals and other venues for publications.
How would you assess the scientific quality of the publications in terms of coherence of argument, methodology and overall analysis?
How would you assess the empirical contributions of the publications?
How would you assess the analytical and/or theoretical contributions of the publications?
If relevant: To what extent do the publications contribute to interdisciplinary research?
How would you assess the originality of the publications within its field, nationally/internationally?
If relevant: To what extent do the publications include co-authoring with significant researchers on the field, nationally and internationally?
How would you assess overall coherence of the research group publication output, that is, the level of shared thematic, theoretical and/or methodological foci in the group?
To what extent do the publications reflect the stated thematic, theoretical and/or methodological foci of the research group?
Overall evaluation of the quality of publication output (reasoning and assessment scale for scientific quality)
178
Appendix L: Societal impact: good practice cases
Case Reasoning
NHH, LEMO-CIO: Media Has substantial underpinning research, clear channels of impact in a variety of venues (the competition commission, private sector consulting, public debate), and some good actual influence is claimed either on a specific decision or in terms of shaping debate.
UiO, Economics: Environmental Economics, and Taxation of Oil Companies
In both cases, there is a clear thread from research-based arguments published in scientific journals and books, through committee and other work, to actual changes to government policy. Although the main immediate impact is above all on Norwegian policies, the impact goes beyond national borders.
UiO, Oslo Fiscal Studies: Childcare An impressive report on how studies of public childcare in Norway (using register data) facilitate the identification of various short and long-run effects on both parents and children. The study points to the importance of universal child care programmes in explaining differences in earnings inequality and income mobility, and it has influenced various policy documents in Norway, as well as the debate in other countries, and also documents by the World Bank, UN and IMF.
UiO, Frisch Centre: Green Tax The case describes the impact of a report on green taxation to which researchers from the Frisch Centre have contributed. The report draws on past research at the centre, in particular in the domain of
environmental (and climate) economics. The case shows that the report was frequently used in the debate on green taxation in the Norwegian parliament; in future, this report may also affect decision-making on green taxation. The impact may even go beyond Norway.
The Research Council of Norway Visiting address: Drammensveien 288 P.O. Box 564
NO–1327 Lysaker
Telephone: +47 22 03 70 00 post@rcn.no
www.rcn.no Publisher:
© The Research Council of Norway www.rcn.no
June 2018
ISBN 978-82-12-03694-9 (pdf) Design: Melkeveien Designkontor AS Photos: Shutterstock
Translation by: Allegro språktjenester AS