• Ei tuloksia

International evaluation of research and doctoral training at the University of Helsinki 2005-2010

N/A
N/A
Info
Lataa
Protected

Academic year: 2022

Jaa "International evaluation of research and doctoral training at the University of Helsinki 2005-2010"

Copied!
383
0
0

Kokoteksti

(1)

AGRICUL TUR AL

AGRIC UL TURAL

AGRICUL TURAL

AGRICULTURAL

AUT ONOMY AND

AUT ONO M Y AND BIBLIOMETR ICS

BIBLIOMETRICS BIBLIOMETRICS

BIBLIOMETRICS BO

TT O M-UP HUMA NITIES

BO TT O M-UP

COMMUNITY

COMMUNITY

CROSSING TRADITIONAL BORDERS

CROSSING TRADITIONAL BORDERS

CROSSING TRADITIONAL BORDERS

CROSSING TRADITIONAL BORDERS

CROSSING TRADITIONAL BORDERS DOCT ORAL TRAI NING

DOCT ORAL TRAINING DOC TORAL TRAINING ENHANCEMENT -LED EV ALU ATION

ENHANCEMENT-LED EVALUA TION

ENHANCEMENT-LED EVALUATION

FIVE INTERNA TION AL PANELS

INTERNA TIONAL

FIVE INTER NA TIONAL P ANELS

FIELD ADJUSTED BIBLIOMETRIC ANAL YSIS FIELD AD JUSTED

BIBLIOMETRIC ANALYSIS

FIELD ADJUSTED BIBLIOMETRIC ANALYSIS

FIELD ADJUSTED BIBLIOMETRIC ANAL YSIS SOCIAL IMP ACT

SOCIAL IMPACT

SOCIAL IMP ACT SOCIAL IMP ACT

HIGH QUALITY RESEARCH

H IG H Q U ALITY RESEARCH H IG H Q U ALITY RESEARCH

HIGH QUALITY RESEARCH

HIGH QUALITY RESEARCH

TRAININ G

TRAINING

TRANSDISCIPLINARITY

TRAINING TRANSDISCIPLINARITY

VOL UNT AR Y

VOLUNT AR Y

VOLUNTARY

VOLUNT AR Y VOL UNT AR Y

FLE XIBLE STRUCTURES FLEXIBLE STRUCTURES

FLEXIBLE STRUCTURES

HELSINKI MODEL

HELSINKI MODEL

HEALTH SCIENCES

HEAL TH SCIENCES

R ES EA R CH ER ORIENTED R ES E A R CH ER ORI ENTED

SCIENTOMETRICS

PEER REVIEW MUL TIDIMENSIONAL

MU LTIDISCIPLINARITY

FUTURE POTENTIAL

FUTURE POTENTIAL

TRANSDISCIPLINARITY

FLEXIBLE STRUCTURES

SCIENT OMETRICS AL

AGRI

AGRICULTURAL

AU TO NO M CS

S B IBLI O METRI CS

BIBLIOMETRICS

OT TO UM A N

O M- P

CO MM UN IT Y

CROSSING TRADITIONAL BORDERS

CROSSING TRADITIONAL BORD

CR OSS IN G TRADITI O NAL BO RDER S NIN

DOC TO RAL TRAININ G DO

ENHANCEMENT-LED EVALUA A A T

-LED EVALUATION AA P INTERNA TION A

FIVE INTER

FIELD ADJUSTED BIBLI O METRI C ANAL YS IS AD JUSTED

TRIC ANALYSIS

FIELD ADJUSTED BIBLIOMETRIC ANALYSIS

D

SOCIAL IMPACT

SOC P AC PP

QUALITY RESEARCH

ALITY RESEARCH

HIGH QUALITY RESEARCH

TRAININ

RAINING

TRANSDISCIPLINARITY

TRANSDISCIPLINAR

VO

VO LUNT AR Y UNT AR Y

FLE

LE STRUCTURES

HELSINKI MODE

HEALTH SCIENCE R ES EA R CH ER

ACT

EME

NAL

O RIENTE D R ES OR

TOMETRICS

LTIDISCIPLINARIT

POTENTIAL

IAL

NSD SCIPLINARITY

EXIBLE STRUCTURES

L

R NO S BO TT HUM A

M- UP

RDERS

NG

EVALUA

ERNA T PANELS

VE INTE

JUSTED

IC ANALY

ANALYSIS

ACT

TY RESEA

AIN

TRAININGTRANSDISCIPLINARITY

NT A

STRU

CES

MET

LE

RUCTURES

CCRREERRS

OOOO CCCC I AAAAL LLL II MPMPMPM AAAA ACCCC TTTT

ALI LITY T T D

HHH EE LLLS SS I N MM OOOD DD EE LL

M

MENT ENT ENT -LE L LE

GG D VV

F RR OO

DD IS SS CCI I PPPL LI NN AA

O

OT T T TE E E EN N N NT TI IAL

S S

SDI DI DI DIS S SC SC

UNIVERSITY OF HELSINKI – EXCELLENCE FOR SOCIETY – HELSINKI MODEL OF RESEARCH EVALUATION

The Evaluation of Research and Doctoral Training is a new model and innovative endeavour to interpret the diversity of the research of a multidisciplinary university. The present evaluation method can be considered to be enhancement led, supporting a bottom-up approach that begins from the researchers. The voluntary-based evaluation succeeded in attracting 1,059 Principal Investigators, and altogether 5,857 researchers participated in the evaluation.

The participants were Researcher Communities – a new concept to be used for evaluation and to plan future research which crosses faculty and departmental borders and proves the collaboration of researchers. The Researcher Communities chose one of the options of participation categories describing their special status or the character of their research:

• Outstanding

• Close to outstanding

• Exceptional

• Innovative opening

• Societal impact

The University of Helsinki has carried out previous research assessments in the years 1998 and 2005. The planning of the present evaluation started in 2010. The data in the evaluation material covered the years 2005–2010.

One exceptional feature in the evaluation was the two types of bibliometric analyses available to the Panels.

The Helsinki University Library prepared tailored bibliometric figures for the entire University and for the Researcher Communities in Social Sciences, Humanities and Computer Sciences. The publication rankings of the Norwegian and Australian models were applied in the evaluation as well. The library analyses proved its innovativeness in the publication analyses. The University of Leiden provided traditional bibliometric analyses for the University and for the Researcher Communities.

The TUHAT Research Information System provided an excellent opportunity to test the publication metadata stored in the database of the University.

The 50 international Panellists represented the five main fields of sciences. The Panels scored the four main evaluation questions and category fitness using the scale 1–5. The mean of the scores for most of the evaluation objects was four or close to that number.

The performance of most RCs can be considered outstanding or high quality. The performance of the entire University is outstanding or high quality when compared to the international field-normalised indicators.

The evaluation results with its recommendations offer tools for the strategic planning of the University.

IN TE RN AT IO N A L E VA LU AT IO N O F R ES EA RC H A N D D O CT O RA L T RA IN IN G A T T H E U N IV ER SIT Y O F H EL SIN KI 20 05 –2 010

INTERNATIONAL EVALUATION OF

RESEARCH AND DOCTORAL TRAINING

AT THE UNIVERSITY OF HELSINKI 2005–2010

Seppo Saari & Antti Moilanen (Eds.)

Se pp o S aa ri & A ntt i M oila ne n ( Ed s.)

ISBN 978-952-10-7557-5 (Paperback) ISBN 978-952-10-7558-2 (PDF) ISSN 1795-5408 (Print) ISSN 1795-5513 (Online)

(2)

RESEARCH AND DOCTORAL TRAINING AT THE UNIVERSITY OF HELSINKI 2005–2010

SEPPO SAARI & ANTTI MOILANEN (EDS.)

University of Helsinki Administrative Publications 81

Evaluations

(3)

ISBN 978-952-10-7557-5 (Paperback) ISBN 978-952-10-7558-2 (PDF) ISSN 1795-5408 (Print) ISSN 1795-5513 (Online) Layout Unigrafia

Printed by Unigrafia, Helsinki 2012 www.helsinki.fi/julkaisut

(4)

ForEword . . . . 11

PAnEl MEMbErs oF biologicAl, AgricUltUrAl And VEtErinAry sciEncEs . . . . 14

PAnEl MEMbErs oF MEdicinE, bioMEdicinE And HEAltH sciEncEs . . . . 16

PAnEl MEMbErs oF nAtUrAl sciEncEs . . . .17

PAnEl MEMbErs oF HUMAnitiEs . . . . 18

PAnEl MEMbErs oF sociAl sciEncEs . . . . 19

oFFiciAls . . . .20

AcronyMs, AbbrEViAtions And dEFinitions APPliEd in tHE rEPort . . . . 21

UniVErsity oF HElsinki in briEF . . . . 25

UniVErsity oF HElsinki in globAl rAnkings . . . . 26

1 . introdUction to tHE EVAlUAtion . . . . 27

1.1 University aims 29 1.2 Steering group and its mandate 29 1.3 Evaluation method 30 1.4 Monetary rewards of the evaluation 32 1.5 Aims and objectives in the evaluation 32 1.6 Conditions to form a Researcher Community for the evaluation 33 1.7 Participation categories 35 1.8 Evaluation material 35 1.9 Evaluation questions, aspects and material 38 1.10 Evaluation feedback and criteria 41 1.11 Timetable of the evaluation 46 2 . iMPlEMEntAtion oF ExtErnAl EVAlUAtion . . . . 47

2.1 Five evaluation Panels 49 2.2 Evaluation related tasks before the Panel meetings in Helsinki 50 2.3 Reporting aims of the Panels 52 2.4 RC-specific evaluation reports 52 2.5 University level report 53 3 . PAnEl-sPEciFic FEEdbAck − biologicAl, AgricUltUrAl And VEtErinAry sciEncEs . . . . 55

3.1 Doctoral training 62 3.2 Development of the RC concept 63 3.3 Distribution of scores 66 3.4 Publication statistics 68 3.5 Bibliometric indicators 69 2. IMPLEMENTATION OF THE EXTERNAL EVALUATION . . . 47

(5)

4.4 Core facilities and platforms for various central methods in campuses 81 4.5 Main aspects that the Panel considered important in the evaluation of RCs 82

4.6 Distribution of scores 83

4.7 Bibliometric indicators 86

5 . PAnEl-sPEciFic FEEdbAck − nAtUrAl sciEncEs . . . . 89

5.1 Introduction 95

5.2 Research 97

5.3 Recommendations to enhance quality and focusing in the research 100

5.4 Practices and quality of the doctoral training 101

5.5 Strategic issues and recommendations regarding practices and

quality in the doctoral training 102

5.6 Research Communities – a unit for new research opportunities and

doctoral training 102

5.7 Distribution of scores 104

5.8 Publication statistics 106

5.9 Bibliometric indicators 107

6 . PAnEl-sPEciFic FEEdbAck − HUMAnitiEs . . . .113

6.1 Doctoral programmes 119

6.2 Financing of doctoral programmes 120

6.3 Internationalization of doctoral programmes 120

6.4 The need for funded post-doctoral positions 121

6.5 Research excellence 121

6.6 The need for risk-taking RCs 122

6.7 Support for the writing of grant proposals and

the development of research agenda’s 123

6.8 Societal impact 123

6.9 Concern for the work load of senior staff 123

6.10 Distribution of scores 124

6.11 Publication statistics 125

6.12 Publication indicators 126

7 . PAnEl-sPEciFic FEEdbAck − sociAl sciEncEs . . . . 133

7.1 Introduction 139

7.2 Research 140

7.3 Doctoral Training 141

7.4 Internationalization 142

7.5 Multidisciplinarity 142

7.6 Societal Impact 143

7.7 Bibliometrics and Statistics 143

7.8 Comment on Categorization 144

7.9 Distribution of scores 145

7.10 Publication statistics 147

7.11 Publication indicators 149

7.12 Publication statistics of the CWTS/Leiden – Social Sciences 155

(6)

8.3 Evaluation scores 168

8.4 Publications of the entire university 169

9 . rEsEArcH PErForMAncE oF tHE UniVErsity oF HElsinki (2005–2010) . . . . 173

9.1 Executive summary 175

9.2 Introduction 176

9.3 Data collection 176

9.4 Methodology 176

9.5 Main results 185

9.6 Results by discipline 192

9.7 Conclusions 198

9.8 Full profiles of University of Helsinki and its areas 199 10 . biblioMEtric AnAlysEs by tHE HElsinki UniVErsity librAry − HUlib . . . .259

10.1 Background 261

10.2 University level bibliometrics by the Helsinki University Library 263

10.3 Faculties, publications and scientific fields 268

10.4 Bibliometric analyses: Faculties, publications and scientific fields 273 10.5 Researcher Communities in HULib bibliometric analyses 286 11 . sUMMAry rEPort on doctorAl stUdEnts’ And PrinciPAl inVEstigAtors’

doctorAl trAining ExPEriEncEs . . . .289

11.1 Introduction 291

11.2 Participants and data collection 292

11.3 Doctoral thesis process 294

11.4 Doctoral studies 303

11.5 Challenges in the development of doctoral education at the university of helsinki 304 11.6 Career and employment survey for University of Helsinki PhD holders 306

11.7 Summary of results 307

12 conclUding rEMArks . . . .309 13 APPEndicEs . . . . 315 11.5. Challenges in the development of doctoral education at the University of Helsinki 304

(7)

3 . . . .

4 . . . . 5 . . . .

6 . . . . 7 . . . .

9 . . . .

Table 3. Biological, Agricultural and Veterinary Sciences: publications, citations and

field-normalized figures in CWTS analysis (2005–2010) 69

Table 4. SUVALUE and VITRI in Norwegian and Australian journal rankings 72 Table 5. SUVALUE and VITRI: weighted values (coefficient) in Norwegian and

Australian journal rankings 72

Table 6. Numeric evaluation of RCs in the Panel of Medicine, Biomedicine and Health Science 83 Table 7. Medicine, Biomedicine and Health Sciences: publications, citations and

field-normalized figures in the CWTS/Leiden analysis (2005−2010) 86 Table 8. Distribution of marks in the Panel of Natural Sciences by participation category

and evaluation parameter 97

Table 9. Numeric evaluation of RCs in the Panel of Natural Sciences 104 Table 10. Natural Sciences: publications, citations and field-normalized figures in

the CWTS/Leiden analysis 107

Table 11. Australian Conference Rankings of ALKO, NODES and SOFTSYS 110 Table 12. Publish or Perish: refereed articles in conference publications of ALKO, NODES and SOFTSYS 110

Table 13. HLG in Norwegian and Australian journal ranking 111

Table 14. HLG’s weighted values (coefficients) in Norwegian and Australian journal rankings 111

Table 15. Numeric evaluation of RCs in the Panel of Humanities 124

Table 16. Publications and indicators of Humanities 125

Table 17. Publications and indicators of Humanities 127

Table 18. Publication performance of RCs in relation to Norwegian and Australian Publication Rankings 130 Table 19. Numeric evaluation of the RCs in the Panel of Social Sciences 145

Table 20. Publications in the Social Sciences 147

Table 21. Social Sciences – publication statistics, Norwegian and Australian Publication Rankings 149 Table 22. Publication performance of the RCs in relation to Norwegian and

Australian Publication Rankings 152

Table 23. Social Sciences: publications, citations and field-normalized figures in

the CWTS/Leiden analysis 155

Table 24. Number of participating Researcher Communities (RCs) as per category and field of science 161 Table 25. Participation activity of principal investigators (PI) in the Faculties of the University 162 Table 26. Participation activity of principal investigators (PI) in

the Independent Institutes of the University 162

Table 27. Number of researchers organised according to tenure track 162 Table 28. Number of RCs in focus areas of research organised according to evaluation panels 165 Table 29. RCs’ focus areas of research in percentage according to evaluation Panels 166 Table 30. Number of focus areas of research in participation categories 166 Table 31. Percentages of focus areas of research in participation categories 167 Table 32. Average scores and standard deviations in the evaluation panels 168 Table 33. Average scores (standard deviations) in the evaluation panels 168

Table 34. Average scores (number of RCs) in categories 169

Table 35. Publication types of all publications under evaluation in the years 2005–2010 169

Table 36. Number of publications by type in the evaluation panels 170

Table 37. Percentages of publications by type in the evaluation panels 170 Table 1. Overview of the bibliometric indicators discussed in this chapter 177 Table 2: Bibliometric data for the publications of a hypothetical research group. 180

Table 3: Overall bibliometric statistics UH 2005–2010 185

Table 4: Trend analysis bibliometric performance indicators UH 185

Table 5: Overview of basic statistics by Discipline and UH research area 187 Table 6: General statistics for Biological, agricultural and veterinary science 193

Table 7: General statistics for Humanities 194

Table 8: General statistics for Medicine, biomedicine and health sciences 195

Table 9: General statistics for Natural sciences 197

Table 10: General statistics for Social sciences 198

(8)

Table 3. Previous figure as a table: the number of authors in publications 265 Table 4. The top 20 scientific journals that have published peer-reviewed

scientific articles have been counted and ranked according to the Finnish

Publication Forum ranking list suggestion (12.1.2012) and Norway journal ranking list 267

Table 5. Number of publications per faculties 268

Table 6. Publication types, number of publications per faculties 269

Table 7. Number of authors in publications and faculties 271

Table 8. Language of publications and faculties 272

Table 9. Number of the RCs’ publications according to the Norwegian and Australian publication rankings in Biological, Agricultural and

Veterinary Sciences, Natural Sciences and Humanities 286

Table 10. Number of the RCs’ publications according to the Norwegian and

Australian publication rankings in Social Sciences 287

Table 1. Membership in doctoral program and research group, and form of the thesis

in each faculty 294

Table 2. Doctoral students’ and PIs’ perceptions of selection criteri 295 Table 3. Reasons for applying for doctoral training (min=1, max=5) 295 Table 4. Doctoral students’ and PIs’ perceptions of main supervisor 299 Table 5. Percentage of doctoral students’ and PIs’ perceptions of frequency of supervision 299 Table 6. Doctoral students’ perceptions of the significance of different factors in

the dissertation project 300

Table 7. Doctoral students’ and PIs’ perceptions of the importance of different persons/groups

in supervision 300

Table 8. Satisfaction with supervision and consideration of changing supervisors in different facultie 302 Table 1. Stages of evaluation including the material provision, timing and responsible actors

in the evaluation 318

Table 2. Evaluation aspects in connection with the evaluation questions. 320

11 . . . .

13 . . . . .

(9)

3 ...

4 ...

5 ...

6 ...

7 ...

8 ...

9 ...

Veterinary Sciences 67

Figure 3. Biological, Agricultural and Veterinary Sciences: Number of WoS and

A1−A4 publications (TUHAT), number of RCs 27 68

Figure 4. Biological, Agricultural and Veterinary Sciences: Number of citations, number of RCs 27 68

Figure 5. MNJS in relation to MNCS 70

Figure 6. THCP10 in relation to MNCS 71

Figure 7. Distributions of numeric evaluation in the Panel of Medicine, Biomedicine and Health Sciences 84 Figure 8 Medicine, Biomedicine and Health Sciences: Number of WoS and

A1-A4 publications, number of RCs 23 85

Figure 9. Medicine, Biomedicine and Health Sciences: Number of citations, number of RCs 23 85

Figure 10. MNJS in relation to MNCS 87

Figure 11. THCP10 in relation to MNCS 88

Figure 12. Distributions of numeric evaluation in the Panel of Natural Sciences 105 Figure 13. Natural Sciences: Number of WoS and A1−A4 publications (TUHAT), RCs 22 106

Figure 14. Natural Sciences: Number of WoS citations, RCs 22 106

Figure 15. MNJS in relation to MNCS 108

Figure 16. THCP10 in relation to MNCS 109

Figure 17. Distributions of numeric evaluation of RCs in the Panel of Humanities 125 Figure 18. Norwegian publication ranking application in Humanities 129 Figure 19. Distribution of the publication performance of RCs in relation to Norwegian and

Australian Publication Rankings 132

Figure 20. Distributions of numeric evaluation of RCs in the Panel of Social Sciences 146

Figure 21. Norwegian publication ranking application 151

Figure 22. Distribution of the publication performance of RCs in relation to Norwegian and

Australian Publication Rankings 154

Figure 23. Social Sciences: number of RC’s publications (P) and total citations (TCS)

in the CWTS/Leiden analysis 155

Figure 24. MNJS in relation to MNCS 156

Figure 25. THCP10 in relation to MNCS 157

Figure 26. Number of RCs in focus areas by evaluation panels 165

Figure 27. Number of RCs in focus areas and categories 167

Figure 28. Number of publications in TUHAT from 4/2010 to 4/2011 (12.4.2011) 171

Figure 29. Number of UH publications in TUHAT 2005-2010 171

Figure 1. Distribution and impact of UH collaboration 186

Figure 2: Overview of 35 UH areas regarding P and MNCS (2005–2010) 189 Figure 3: Overview of 35 UH areas regarding P and MNCS (2005–2010) 190 Figure 4: Overview of 35 UH areas regarding P and MNCS (2005–2010) 191 Figure 5: Performance of all Biological, Agricultural and veterinary sciences areas

with the entire landscape 192

Figure 6: Performance of all Humanities areas with the entire landscape 194 Figure 7: Performance of all Medicine, Biomedicine and Health sciences areas

within the entire landscape 195

Figure 8. Performance of all Natural Sciences areas within the entire UH landscape 196 Figure 9: Performance of all Social Sciences areas within the entire UH landscape 197

(10)

Figure 4. Publications by subject – journal articles 266

Figure 5. Percentages of publication types by faculties 270

Figure 6. Number of publications within subject areas of the Faculty of Agriculture and Forestry 274 Figure 7. Number of publications within subject areas of the Faculty of Arts 275 Figure 8. Number of publications within subject areas of the Faculty of Behavioural Sciences 276 Figure 9. Number of publications within subject areas of the Faculty of Biological and

Environmental Sciences 277

Figure 10. Number of publications within subject areas of the Faculty of Law 278 Figure 11. Number of publications within subject areas of the Faculty of Medicine 279 Figure 12. Number of publications within subject areas of the Faculty of Pharmacy 280 Figure 13. Number of publications within subject areas of the Faculty of Science 281 Figure 14. Number of publications within subject areas of the Faculty of Social Sciences 282 Figure 15. Number of publications within subject areas of the Faculty of Theology 283 Figure 16. Number of publications within subject areas of the Faculty of Veterinary Medicine 284 Figure 17. Number of publications within subject areas of the Swedish School of Social Sciences 285 Figure 1. Facilitating factors according to doctoral students and PIs 296 Figure 2. Impeding factors experienced by doctoral students and PIs 297 Figure 3. Doctoral students’ and PIs’ perceptions of the most important tasks of a supervisor.

The relation was statistically significant 301

Figure 4. Primary work assignments reported by PhD holders 307

11 ...

(11)

Appendix 2. Detailed time table of the Evaluation 319

Appendix 3. Evaluation aspects 321

Appendix 4. Table of Fields of sciences in the CWTS/Leiden analyses 322

Appendix 5. Stage 1, e-form for registration 328

Appendix 6. Stage 2, e-form for submission of evaluation material 332

Appendix 7. Evaluation Form 337

Appendix 8. Figure of completed doctoral degrees at the University of Helsinki 2000−2011 344 Appendix 9. Tables of scores of participating RCs by participation category 345 Appendix 10. Distributions of numeric evaluation of the RCs in the evaluation panels 348 Appendix 11. Number of publications in the evaluation (TUHAT RIS) 353 Appendix 12. Tables of RC-specific bibliometric indicators by the CWTS/Leiden 358

Appendix 13. PhD student survey 362

Appendix 14. PI Survey 369

Appendix 15. Differences in reasons for conducting a PhD between students who have

considered interrupting their studies and students who have not considered interrupting 376 Appendix 16. Satisfaction and intentions to interrupt studies in different faculties 376 Appendix 17. PIs and students’ perceptions of the emphasis of different kind of supervision in

the beginning of the doctoral process and later 377

Appendix 18. Sum variables and differences in sum variables between the faculties 378 Appendix 19. Candidates’ and PIs perceptions of studies required for doctoral degree 380

Appendix 20. Graduation times in different faculties 380

(12)

FOREWORD

The evaluation of research and doctoral training was carried out in the years 2010–2012. The steering group appointed by the Rector in January 2010 set the conditions for participation in the evaluation and prepared the Terms of Reference to present the evaluation procedure and criteria. The publications and other scientific activities included in the evaluation covered the years 2005–2010.

The participating unit in the evaluation was defined as a Researcher Community (RC). To obtain a critical mass with university level impact, the number of members was set to range from 20 to 120. The RCs were required to contain researchers in all stages of their research career, from doctoral students to principal investigators (PIs). All in all, 136 Researcher Communities participated in this voluntary evaluation, 5857 persons in total, of whom 1131 were principal investigators. PIs were allowed to participate in two communities in certain cases, and 72 of them used this opportunity.

This evaluation enabled researchers to define RCs from the “bottom up” and across disciplines. The aim of the evaluation was not to assess individual performance but a community with shared aims and researcher-training activities. The RCs were able to choose among five different categories that characterised the status and main aims of their research. The steering group considered the process of applying to participate in the evaluation to be important, which lead to the establishment of these categories. In addition, providing a service for the RCs to enable them to benchmark their research at the global level was one of the main goals of the evaluation.

The data for the evaluation consisted of the RCs’ answers to evaluation questions on supplied e-forms and a compilation extracted from the research information system (RIS) TUHAT on 12 April 2011. The compilation covered scientific and other publications as well as certain areas of scientific activities. During the process, the RCs were asked to check the list of publications and other scientific activities and to make corrections if needed. These TUHAT compilations are public and available on the evaluation project sites of each RC in the TUHAT system.

In addition to the e-form and TUHAT compilation, the University of Leiden (CWTS) carried out bibliometric analyses of the articles included in the Web of Science (WoS).

This was done according to the fields of sciences and levels of the RCs. In cases where most of the publications of the RC were not included in the WoS data, the Library of the University of Helsinki conducted a separate analysis of the publications. This was done for 66 RCs, mainly representing the humanities and social sciences. The total publications of the university were also analysed by the Helsinki University Library, and this analysis is a part of this report.

The evaluation office also carried out an enquiry targeted to PhD candidate supervisors and PhD candidates about the organisation of doctoral studies at the University of Helsinki. This and other documents describing the University and the Finnish higher education system were provided to the panellists.

(13)

The panel feedback for each RC is unique and presented as an entity in each panel- specific report. The first collective evaluation reports available to the entire panel were prepared in July–August 2011. The reports were accessible to all panel members via the electronic evaluation platform in August. Scoring from 1 to 5 was used to complement written feedback in association with evaluation questions 1–4 (scientific focus and quality, doctoral training, societal impact, cooperation), in addition to the category evaluating the fitness for participation in the evaluation. The panellists used the international level as a point of comparison in the evaluation. Scoring was not expected to follow a preset normal deviation.

Each of the draft reports were discussed and dealt with by the panel in meetings in Helsinki (from 11 September to 13 September or from 18 September to 20 September 2011). In these meetings the panels also examined the deviations among the scores and finalised the draft reports together.

This university level report deals shortly with the background of the evaluation and the terms of participation. The main evaluation feedback is provided in the RC- specific evaluation reports, which are published as 136 separate reports. Summaries and central findings as well as the main recommendations written by each panel are published in the university report. The key figures of the bibliometric analyses are presented in this report.

On behalf of the evaluation steering group and office, I sincerely wish to thank all the RCs warmly for your participation in this evaluation. The effort you made in submitting the data to TUHAT-RIS is gratefully acknowledged by the University. We hope that you find this panel feedback useful in many ways. The bibliometric profiles may open a new view on publication forums and provide a perspective for discussion on the choice of forums. We especially hope that this evaluation report will help the University in setting future goals for research.

Johanna Björkroth Vice-Rector

Chair of the Steering Group of the Evaluation

(14)

STEERING GROUP OF THE EVALUATION

Steering group, nominated by the Rector of the University, was responsible for the planning of the evaluation and its implementation having altogether 22 meetings between February 2010 and March 2012.

CHAIR

Vice-Rector, professor Johanna Björkroth

VICE-CHAIR

Professor Marja Airaksinen

Chief Information Specialist, Dr Maria Forsman Professor Arto Mustajoki

University Lecturer, Dr Kirsi Pyhältö

Director of Strategic Planning and Development, Dr Ossi Tuomi Doctoral candidate, MSocSc Jussi Vauhkonen

The Panel members are introduced on the following pages

The panels independently evaluated all the submitted material and were responsible for the feedback for the RC-specific reports. The panels wrote a university level summary based on all the material discussed in the panel. The panel members were asked to confirm whether they had any conflicts of interest with the RCs. If this was the case, the panel members disqualified themselves from the discussion and report writing.

(15)

PANEL MEMBERS OF BIOLOGICAL, AGRICULTURAL AND VETERINARY SCIENCES

CHAIR

Professor Ary A. Hoffman

Ecological genetics, evolutionary biology, biodiversity conservation, zoology University of Melbourne, Australia

VICE-CHAIR

Professor Barbara Koch Forest Sciences, remote sensing University of Freiburg, Germany Professor Per-Anders Hansson

Agricultural engineering, modeling, life cycle analysis, bioenergy Swedish University of Agricultural Sciences, Sweden

Professor Danny Huylebroeck Developmental biology

Katholieke Universiteit Leuven, Belgium Professor Jonathan King

Virus assembly, protein folding

Massachusetts Institute of Technology MIT, USA Professor Hannu J.T. Korhonen

Functional foods, dairy technology, milk hygiene MTT Agrifood Research Finland

Professor Kristiina Kruus

Microbiological biotechnology, microbiological enzymes, applied microbiology VTT Technical Research Centre of Finland

Professor Joakim Lundeberg

Biochemistry, biotechnology, sequencing, genomics KTH Royal Institute of Technology, Sweden Professor Dominiek Maes

Veterinary medicine Ghent University, Belgium Professor Olli Saastamoinen Forest economics and policy University of Eastern Finland Professor Kai Simons

Biochemistry, molecular biology, cell biology

Max-Planck-Institute of Molecular Cell Biology and Genetics, Germany

(16)

Added expertise to the evaluation was contributed by members from other panels and by one evaluator outside the panels.

External Expert

Professor Anders Linde Oral biochemi Faculty of Odontology Göteborg University, Sweden

Experts from Other Panels

Professor Caitlin Buck, from the Panel of Natural Sciences Professor Ritske Huismans, from the Panel of Natural Sciences

Professor Johanna Ivaska, from the Panel of Medicine, Biomedicine and Health Sciences Professor Lea Kauppi, from the Panel of Natural Sciences

Professor Holger Stark, from the Panel of Natural Sciences

Professor Peter York, from the Panel of Medicine, Biomedicine and Health Sciences

(17)

PANEL MEMBERS OF MEDICINE, BIOMEDICINE AND HEALTH SCIENCES CHAIR

Professor Lorenz Poellinger

Cancer biology, cell and molecular biology Karolinska Institute, Sweden

VICE-CHAIR

Professor Cornelia van Duijn

Genetic epidemiology, Alzheimer’s disease and related disorders Erasmus Medical Centre, the Netherlands

Professor Johanna Ivaska

Molecular cell biology, cell adhesion, cancer biology University of Turku, VTT Technical Research Centre, Finland Professor Olli Lassila

Immunology, medical microbiology University of Turku, Finland Professor Hans-Christian Pape Neuroscience, neurophysiology University of Münster, Germany Professor Thomas Ruzicka Dermatology, allergology

Ludwig-Maximilians-Universität (LMU) München, Germany Professor Lars Terenius

Experimental alcohol and drug dependence research, mental disorders, preventive medicine Karolinska Institute, Sweden

Professor Peter York

Physical pharmaceutics, pharmaceutical chemistry, pharmaceutical technology University of Bradford, Great Britain

Added expertise to the evaluation was contributed by two evaluators outside the Panels and by three members from other Panels.

External Experts

Professor Olli Carpén

Pathology, cancer cell metastasis University of Turku, Finland Professor Anders Linde Oral biochemi Faculty of Odontology Göteborg University, Sweden

Experts from Other Panels

Professor Jan-Otto Carlsson, from the Panel of Natural Sciences

Professor Danny Huylebroek, from the Panel of Biological, Agricultural and Veterinary Sciences Professor Holger Stark, from the Panel of Natural Sciences

(18)

PANEL MEMBERS OF NATURAL SCIENCES CHAIR

Professor Jan-Otto Carlsson

Materials science in chemistry and physics, nanotechnology, inorganic chemistry Uppsala University, Sweden

VICE-CHAIR

Professor Jan van Leeuwen

Computer science, information technology University of Utrecht, the Netherlands Professor Caitlin Buck

Probability and statistics, archeology, palaeoenvironmental science University of Sheffield, Great Britain

Professor David Colton

Mathematics, inverse problems of acoustic and electromagnetic scattering University of Delaware, USA

Professor Jean-Pierre Eckmann

Mathematics, dynamical systems, mathematical physics University of Geneva, Switzerland

Professor Ritske Huismans Geosciences, geodynamics University of Bergen, Norway Professor Jukka Jurvelin Medical physics and engineering University of Eastern Finland Professor Lea Kauppi

Environmental sciences, water research The Finnish Environment Institute, Finland Professor Riitta Keiski

Chemical engineering, heterogeneous catalysis, environmental technology, mass and heat trans- fer processes

University of Oulu, Finland Professor Mats Larsson

Experimental molecular physics, chemical dynamics, molecular spectroscopy, astrobiology Stockholm University, Sweden

Professor Holger Stark

Medicinal, organic and pharmaceutical chemistry, pharmacology Johann Wolfgang Goethe Universität, Germany

Added expertise to the evaluation was contributed by the members from other Panels.

Experts from Other Panels

Professor Barbara Koch, from the Panel of Biological, Agricultural and Veterinary Sciences Professor Peter York, from the Panel of Medicine, Biomedicine and Health Sciences

(19)

PANEL MEMBERS OF HUMANITIES CHAIR

Professor Wim van den Doel

Contemporary history, history of European relations with the world beyond Europe Leiden University, the Netherlands

VICE-CHAIR

Professor Kerstin Jonasson Romance languages, linguistics Uppsala University, Sweden Professor Regina Bendix

European ethnology, scientific history of ethnography, folklore University of Göttingen, Germany

Professor Paul Cobley

History, American studies, communication, semiotics London Metropolitan University, Great Britain Professor Troels Engberg-Pedersen1

Theology, early Christian thought, ancient philosophy University of Copenhagen, Denmark

Professor Erhard Hinrichs

Linguistics, language technology, infrastructures Eberhard Karls Universität Tübingen, Germany Professor Jutta Scherrer

Intellectual and cultural history of Russia, history of ideologies L’École des hautes études en sciences sociales (EHESS), France Professor Klaus Tanner2

Theology, ethics

University of Heidelberg, Germany Professor Pauline von Bonsdorff Aesthetics, art education University of Jyväskylä, Finland

Added expertise to the evaluation was contributed by the members from other Panels.

Experts from Other Panels

Professor Caitlin Buck, from the Panel of Natural Sciences Professor Allen Ketcham, from the Panel of Social Sciences Professor Erno Lehtinen, from the Panel of Social Sciences Professor Jan van Leeuwen, from the Panel of Natural Sciences

1 Professor Engberg-Pedersen contributed in the report writing although he was not able to take part in the meetings in Helsinki.

2 Professor Tanner was involved in the discussions in Helsinki, but not in the pre-work and report writing.

(20)

PANEL MEMBERS OF SOCIAL SCIENCES CHAIR

Professor Hebe Vessuri Social anthropology

Venezuelan Institute of Scientific Research, Venezuela

VICE-CHAIR

Professor Christine Heim

Psychology, neurobiology of early-life stress, depression, anxiety, functional somatic disorders Charité University Medicine Berlin, Germany

Professor Allen Ketcham

Ethics and social philosophy, applied Social philosophy, ethics of business Texas A&M University – Kingsville, USA

Professor Erno Lehtinen Education, educational reform University of Turku, Finland Professor Enzo Mingione Urban sociology

University of Milan - Bicocca, Italy Professor Giovanna Procacci

Political sociology, transformation of citizenship, social rights, social exclusion, immigration policy University of Milan, Italy

Professor Inger Johanne Sand Law, public law, legal theory University of Oslo, Norway Professor Timo Teräsvirta Time series econometrics Aarhus University, Denmark Professor Göran Therborn General sociology

University of Cambridge, Great Britain Professor Liisa Uusitalo

Consumer behaviour (economic & social theory), marketing and communication research Aalto University, School of Economics, Finland

Added expertise to the evaluation was contributed by two members from the Panel of Humani- ties.

Experts from the Panel of Humanities

Professor Erhard Hinrichs Professor Pauline von Bonsdorff

(21)

OFFICIALS

EVALUATION OFFICE Editors of the reports

Dr Seppo Saari, Doc., Senior Adviser in Evaluation, was responsible for the entire evaluation, its planning and implementation and acted as an Editor-in-chief of the reports.

Mr Antti Moilanen, Project Secretary, was responsible for editing the reports. He worked in the evaluation office from January 2012 to May 2012.

MSocSc Paula Ranne, Planning Officer, was responsible for organising the panel meetings and all the other practical issues like agreements and fees and editing a part the RC-specific reports. She worked in the evaluation office from March 2011 to January 2012.

Dr Eeva Sievi, Doc., Adviser, was responsible for the registration and evaluation material compila- tions for the panellists. She worked in the evaluation office from August 2010 to July 2011.

TUHAT OFFICE - Provision of the publication and other scientific activity data

Mrs Aija Kaitera, Project Manager of TUHAT RIS served the project ex officio providing the evalua- tion project with the updated information from TUHAT RIS. The TUHAT office assisted in mapping the publications with the CWTS/University of Leiden.

MA Liisa Ekebom, Assisting Officer, served in TUHAT RIS updating the publications for the evalua- tion. Ekebom also assisted the UH/Library analyses (Spring and Autumn 2011)

BA Liisa Jäppinen, Assisting Officer, served in TUHAT RIS updating the publications for the evalu- ation (Spring 2011)

HELSINKI UNIVERSITY LIBRARY - Provision of the publication analyses

Dr Maria Forsman, Chief Information Specialist in the Helsinki University Library, managed with her 10 colleagues the bibliometric analyses in humanities, social sciences and in other fields of sciences where the CWTS/Leiden analyses were not applicable.

PROVISION OF DOCTORAL SURVEY

Dr Kirsi Pyhältö, Doc. PhD., Senior Lecturer in University Pedagogy, University of Helsinki MA Minna Frimodig, Education Adviser, Rector’s Office, Academic Affairs, University of Helsinki

(22)

ACRONYMS, ABBREVIATIONS AND DEFINITIONS APPLIED IN THE REPORT

External competitive funding

AF – Academy of Finland

TEKES – Finnish Funding Agency for Technology and Innovation EU – European Union

ERC – European Research Council International and national foundations

FP7/6 etc. /Framework Programmes/Funding of European Commission

Evaluation marks

Outstanding (5) Excellent (4) Very Good (3)

Good (2)

Sufficient (1)

Abbreviations of Bibliometric Indicators

P - Number of publications TCS – Total number of citations

MCS – Number of citations per publication, excluding self-citations PNC – Percentage of uncited publications

MNCS – Field-normalized number of citations per publication MNJS – Field-normalized average journal impact

THCP10 – Field-normalized proportion highly cited publications (top 10%)

INT_COV – Internal coverage, the average amount of references covered by the WoS WoS – Thomson Reuters Web of Science® Databases

TUHAT Research Information System (RIS)

Research Information System of University of Helsinki

Explanation of Norwegian, Australian and Finnish Publication Forum (FPF) rankings

(http://www.tsv.fi/julkaisufoorumi/materiaalit/julkaisufoorumi_lehdet_

sarjat_35842_10022011.xls)

Norwegian journal (NJ) and publisher (NP) ranking (http://dbh.nsd.uib.no):

• Level 2 – highest scientific

• Level 1 – scientific.

Australian ranking (http://www.arc.gov.au/era/era_2010/archive/key_docs10.htm):

• A* – Virtually all papers they publish will be of a very high quality

• A – The majority of papers in a Tier A journal will be of very high quality

• B – Generally, in a Tier B journal, one would expect only a few papers of very high quality

• C – Journals that do not meet the criteria of higher tiers.

(23)

Finnish Publication Forum (FPF) rankings

• 3 – top leading scientific journal

• 2 – leading scientific journal

• 1 – scientific journal

Participation category

Category 1. The research of the participating community represents the international cutting edge in its field.

Category 2. The research of the participating community is of high quality, but the community in its present composition has yet to achieve strong international recognition or a clear break-through.

Category 3. The research of the participating community is distinct from mainstream research, and the special features of the research tradition in the field must be considered in the evaluation.

Category 4. The research of the participating community represents an innovative opening.

Category 5. The research of the participating community has a highly significant societal impact.

Research focus areas of the University of Helsinki

Focus area 1. The basic structure, materials and natural resources of the physical world Focus area 2. The basic structure of life

Focus area 3. The changing environment – clean water Focus area 4. The thinking and learning human being Focus area 5. Welfare and safety

Focus area 6. Clinical research Focus area 7. Precise reasoning Focus area 8. Language and culture Focus area 9. Social justice

Focus area 10. Globalisation and social change

Type of publication in TUHAT RIS

Type of classification follows mainly the definition of the Ministry of Education and Culture.

A1 Refereed journal article A2 Review in scientific journal

A3 Contribution to book/other compilations (refereed) A4 Article in conference publication (refereed) B1 Unrefereed journal article

B2 Contribution to book/other compilations (non-refereed) B3 Unrefereed article in conference proceedings

C1 Published scientific monograph

C2 Edited book, compilation, conference proceeding or special issue of journal D1 Article in professional journal

D2 Article in professional hand or guide book or in a professional data system, or text book material

D3 Article in professional conference proceedings D4 Published development or research report

D5 Text book or professional handbook or guidebook or dictionary E1 Popular article, newspaper article

E1 Popular contribution to book/other compilations

(24)

E2 Popular monograph

F1 Published independent artistic work F2 Public contribution to artistic work F3 Public artistic play or exhibition

F4 Model or plan taken into production / exploited G1-G5 Theses (not included in the evaluation)

H1 Patents

I1 Audiovisual materials I2 ICT programs or applications

Type of other scientific activity in TUHAT RIS

Assessment of candidates for academic posts Editor of communication journal

Editor of research anthology/collection/conference proceedings Editor of research journal

Editor of series

Editor of special theme number

Membership or other role in national/international committee, council, board Membership or other role in public Finnish or international organization Membership or other role in research network

Membership or other role in review committee

Membership or other role of body in private company/organisation Other tasks of an expert in private sector

Participation in interview for web based media Participation in interview for written media Participation in radio programme

Participation in TV programme Peer review of manuscripts Prizes and awards

Supervisor or co-supervisor of doctoral thesis

Principal Investigator (PI) at the University of Helsinki (2009)

A principal investigator at the University of Helsinki is typically a person who:

i) steers and leads research independently,

ii) has completed an applicable doctoral degree and become qualified as an independent researcher,

iii) has access to the necessary resources (facilities, funding, equipment) for independent research,

iv) supervises doctoral students and/or mentors post-doctoral researchers as well as (in applicable research fields) leads a research group, and

v) is placed on the third or fourth level in the hierarchy of research positions.

(25)

Affiliation with the University of Helsinki

A person is or has been affiliated with the University of Helsinki, if he or she is or has been employed by the University as a researcher or doctoral candidate between 1 January 2005 and 31 October 2010.

Scholars not employed by the University of Helsinki are nevertheless considered to be affiliated with the University if they have:

• worked at the University as a researcher or doctoral candidate between 1 January 2005 and 31 October 2010, and

• received external funding from a source other than a university or a research institute.

A further prerequisite for the affiliation of doctoral candidates is that they must have been granted the right to pursue postgraduate studies at the University of Helsinki.

A docentship alone does not constitute an affiliation with the University.

(26)

UNIVERSITY OF HELSINKI IN BRIEF

University of Helsinki is 372 years old, established in Turku 1640 and moved to Helsinki 1828.

Main tasks of the University are research, teaching and societal interaction. The University is bilingual, Finnish and Swedish. Several courses in English are provided.

• 37,000 degree students

• 32,000 continuing education and Open University students

• 8,590 employees, including 4,820 researchers and teachers (2011)

• Total funding 648 million euro (2011)

• Operates on four campuses in Helsinki and 14 other locations throughout Finland

• 25 national top research units, one Nordic top research unit, 15 Academy professors and 15 FiDiPro professors. New top research units for the years 2012–2017. 10 of 15 are totally or partially working at the University of Helsinki

• Scientific publications per year from 6,500 to 7,000. All publications 10,000 per year

• Masters degrees 2,200 per year

• Doctoral degrees 430 per year

• Foreign degree students 2,000

Campuses and Faculties

City Center

• Faculty of Theology

• Faculty of Law

• Faculty of Arts

• Faculty of Behavioural Sciences

• Faculty of Social Sciences

• Swedish School of Social Science Kumpula

• Faculty of Science Meilahti

• Faculty of Medicine Viikki

• Faculty of Biological and Environmental Sciences

• Faculty of Pharmacy

• Faculty of Agriculture and Forestry

• Faculty of Veterinary Medicine

Independent institutes

• Aleksanteri Institute - Finnish Centre for Russian and East European Studies

• Center for Information Technology (IT Center)

• Center for Properties and Facilities

• Finnish Museum of Natural History

• Helsinki Center of Economic Research (HECER)

• Helsinki Collegium for Advanced Studies

(27)

• Helsinki Institute for Information Technology

• Helsinki Institute of Physics (HIP)

• Helsinki University Library

• Institute of Biotechnology

• Institute for Molecular Medicine Finland (FIMM)

• IPR University Center

• Ruralia-institute

• Language Centre

• The National Library of Finland

• Helsinki University Laboratory Animal Centre

• Neuroscience Center

• Open University

• Palmenia Centre for Continuing Education

• UniSport

UNIVERSITY OF HELSINKI IN GLOBAL RANKINGS

3

SHANGHAI QS4 TIMES HIGHER

LEIDEN TAIWAN GRONINGEN WEBOMETRICS

2003 74 - - - -

2004 72 129 - - - - -

2005 76 62 - - - - -

2006 74 116 - - - - -

2007 73 100 - - 52 - -

2008 68 91 - - 50 - -

2009 72 108 - 535 48 436 52

2010 72 75 102 - 47 - 64 and 597

2011 74 89 91 - 66 - 62 and 785

Note: hyphen (-) indicates that ranking is not published

The University of Helsinki is ranked high in the comparison of the world’s top uni- versities. On the esteemed Shanghai list UH is ranked 74th and on the Times Higher Education it is 91st.

With some 15,000 universities in the world, the University of Helsinki performs very well, especially when considering Finland’s population base (5,4 million) and resources.

4567

3 Table provided by Markus Laitinen, International Affairs, UH 4 Earlier Times

5 P * CPP/FCSm 2003–2007 6 CPP 1998–2008

7 2010 and 2011 twice a year

(28)

3 PANEL-SPECIFIC FEEDBACK − BIOLOGICAL, AGRICULTURAL AND VETERINARY SCIENCES

4 PANEL-SPECIFIC FEEDBACK − MEDICINE, BIOMEDICINE AND HEALTH SCIENCES

5 PANEL-SPECIFIC FEEDBACK − NATURAL SCIENCES

6 PANEL-SPECIFIC FEEDBACK − HUMANITIES

7 PANEL-SPECIFIC FEEDBACK − SOCIAL SCIENCES

8 OVERALL STATISTICS ON THE EVALUATION

9 RESEARCH PERFORMANCE

10 BIBLIOMETRIC ANALYSES

11 SUMMARY REPORT OF DOCTORAL TRAINING

12 CONCLUDING REMARKS

13 APPENDICES

2 IMPLEMENTATION OF THE EXTERNAL EVALUATION

(29)
(30)

1. INTRODUCTION TO THE EVALUATION

1.1 UNIVERSITY AIMS

The new strategy of the University of Helsinki for the years 2013–2016 sets the strategic objective of the University to be counted among the 50 leading universities in the world by 2020. The goal requires that the University remains at the cutting edge or research in as many fields as possible and that it enhances its reputation as a high-quality learning environment with the ability to resolve global issues. The mission of the University is to be the most comprehensive research institution of higher education, edification and intellectual regeneration in Finland. It is a pioneer and builder of the future. To make right strategic choices in achieving presented goals, the University regularly carries out international evaluations of its research and teaching. Previous research evaluations including doctoral training was carried out in 1999 and 2005. The present international evaluation of research and doctoral training to be launched took place between 2010 and 2012 and the material under evaluation covered the period from 1 January 2005 to 31 December 2010.

The University’s performance depends on its personnel and students. The evaluation findings will be expected to offer useful information to the university to identify areas of strength as well as areas in need of development within research and doctoral training.

Also, structures that relate to research and doctoral training and extend over faculty and departmental boundaries will be more readily recognised. For the Researcher Community, the evaluation offers an opportunity to receive feedback from external experts and to obtain feedback on the quality of research and doctoral training in the international context.

1.2 STEERING GROUP AND ITS MANDATE

For the planning of the evaluation, the Rector of the University appointed a steering group 27 January 2010, Decision No 101/01/2010. The steering group was chaired by Vice-Rector Johanna Björkroth with Professor Marja Airaksinen who served as the Vice-Chair. The other group members were Chief Information Specialist Maria Forsman, Professor Arto Mustajoki, University Lecturer Kirsi Pyhältö, Director of Strategic Planning and Development Ossi Tuomi and doctoral candidate Jussi Vauhkonen.

The steering group prepared the evaluation plan, tasks, aims, objectives, methods, the timetable for implementation and the bases for financial or other recognition from the evaluation. The basic material used in the preparation was the strategic documents of the University. In the planning process, the steering group thoroughly discussed the options for how the evaluation would be implemented. A particular focus of the discussion was how to ensure continuity with the previous evaluations and what

(31)

means could be used to support the choice of research focus areas, societal impact, new innovative research openings, the composition of research communities and their interdisciplinarity, multidisciplinarity or transdisciplinarity, and international visibility. It was stated in the discussions that the new structures in the faculties did not support continuity with the previous evaluations. The steering group argued on behalf of voluntary participation and that the participating Researcher Communities (RCs) should cross department and faculty borders. RC refers to the group of researchers who registered together to participate in the evaluation of their research and doctoral training. It was assumed that this voluntary aspect would support the bottom-up approach in forming Researcher Communities. Further, encouragement factors were also needed, such as benefits for the researcher groups, departments or faculties of the university. The main impetus for using the new method was the strategic aim of the university: “to the top and out to society” or “excellence for society”.

The key topic in the discussions of the steering group was how to be able to recognise the diversity of research and its conditions and preconditions in a multidisciplinary university.

After the planning period, the steering group continued its work and followed the implementation of the evaluation and made principled decisions when necessary.

During the planning stage, the steering group discussed the implementation of the evaluation with the academic community in several forums.

1.3 EVALUATION METHOD Background

The structure of the faculties and departments has changed since the year 2010. The steering group stated that there is no longer a good reason to repeat the previous evaluation model and to compare the research performance of the old and new structures. In practice, the implementation of a new model meant taking a risk. There were many unknown factors, and the university had no earlier experience in how to manage the selected model successfully.

The concept of a Researcher Community was especially unclear to both the academic community and the panellists. Also, the distinctions between categories were not clear, and the categories were not exclusive. The RCs had several options to choose from among the participation categories. It was not, however, possible to foresee or speculate on the optimal choice of category.

Bottom-up approach

The participants in the evaluation were Researcher Communities (RCs). Conditions in forming RC were given in the Guidelines for the Participating Researcher Communities.

The RCs defined whether the composition of their communities should be considered, for example, as well-established or new.

(32)

The challenge for this evaluation was to recognise and justify the diversity of research practices and publication traditions. Traditional Research Assessment Exercises (RAE) do not necessarily value high quality research if the volumes are low or the research is distinct from mainstream research. It is always challenging to expose the diversity of research to fair comparison. Understanding the divergent starting points of the RCs demanded sensitivity from the panellists.

The evaluation’s emphasis on a researcher-oriented approach was already apparent during the registration procedure when the RCs were formed. The platform for the evaluation made it possible to consider a variety of compositions of researcher communities. Because the publications covered the years 2005–2010, it was assumed that the RCs would have certain aspects in common during the years under evaluation, e.g., common themes in their publications or some common type of cooperation and also common plans for the future.

The evaluation can be considered as enhancement led. Instead of ranking, the main aim was to provide useful information for the enhancement of the research and doctoral training of the participating RCs. The comparison should take into account each field of science and acknowledge its special characteristics. The comparison should be able to produce information to identify the present status of the RC and the factors that have lead to success. Moreover, challenges in the operations and outcomes should be recognised.

The evaluation approach was designed to recognise the significance and specific nature of researcher communities and research areas in a multidisciplinary top level university. Furthermore, one of the aims of the evaluation was to bring to light those evaluation principles that differ from the prevalent ones. Thus, the views of various fields of research can be described and research arising from various starting points better understood. The evaluation of doctoral training was integrated into the evaluation as a natural component related to research. Operational processes of doctoral training were examined in the evaluation questions and in a separate doctoral survey for background information.

Five main stages of the evaluation method

• Registration

• Self-evaluation

• TUHAT8 compilations on publications and other scientific activities9

• External evaluation

• Public reporting

The external part of the evaluation – the peer evaluation – took place in panels comprising distinguished national and international experts who based their evaluation on the materials submitted by the participating Researcher Communities and the data stored in the University’s research information system TUHAT.

The previous evaluations of research, in the years 1998 and 2005, covered all the institutions in the university. The previous model was traditional in the sense that a 8 TUHAT (acronym) of Research Information System (RIS) of the University of Helsinki

9 E.g. editorial work, memberships, public appearances, peer reviews, supervision or co-supervision of doctoral thesis

(33)

distinguished collection of research reports formed the main evaluation material in addition to the evaluation questions. The external panels evaluated the publications and scored their level of performance.

It is essential to emphasise that the present evaluation combined both meta-evaluation and traditional research assessment and that its focus was both on research outcomes and the procedures associated with research and doctoral training. The approach to the evaluation where self-evaluation constituted the main source of information can be considered enhancement led. The answers to the evaluation questions together with the information about publications, its bibliometric analyses and the lists of other scientific activities formed an entity that was to be reviewed as a whole.

1.4 MONETARY REWARDS OF THE EVALUATION

The financial consequence of the first research assessment in the year 1998 took the form of monetary rewards to successful departments. Successful faculties were rewarded as well.

In the year 2005, the departments and faculties with high scores whose performance was enhanced compared to the previous evaluation were rewarded. The rewards were assigned for 3 or 6 years.

The Rector will decide on the amount and allocation criteria of the resources to be distributed on the basis of the present evaluation results. High quality performance as well as the amount of participation in the evaluation will be considered in the allocation of resources in the planning of the next strategy period (2013–2016) and in the preparation of the University’s research policy.

1.5 AIMS AND OBJECTIVES IN THE EVALUATION

The aims of the evaluation were stated as follows:

• to improve the level of research and doctoral training at the University of Helsinki and to raise their international profile in accordance with the University’s strategic policies.

The improvement of doctoral training should be compared to the University’s policy,10

• to enhance the research conducted at the University by taking into account the diversity, originality, multidisciplinary nature, success and field-specificity,

• to recognize the conditions and prerequisites under which excellent, original and high- impact research is carried out,

• to offer the academic community the opportunity to receive topical and versatile international peer feedback,

• to better recognize the University’s research potential,

• to exploit the University’s TUHAT research information system to enable transparency of publishing activities and in the production of reliable, comparable data.

10 Policies on doctoral degrees and other postgraduate degrees at the University of Helsinki.

(34)

1.6 CONDITIONS TO FORM A RESEARCHER COMMUNITY FOR THE EVALUATION

The evaluation was targeted to researcher communities which were formed on the basis of collaboration in research and doctoral training. The researcher communities must include Principal Investigators (PI) and doctoral candidates. In addition, the researcher community typically included academics also on the other levels of the four-level hierarchy of researcher positions. As the purpose of the evaluation was to recognise the conditions and requirements for producing cutting edge and high quality research results and doctoral training, the University encouraged researcher communities with established collaboration between its members to participate. The practical motivation (e.g. research, doctoral training) for forming the researcher community was to be demonstrated in the evaluation materials.

Researcher communities that, in addition to meeting the above requirements, had to fulfil the following conditions (a-c) in the evaluation:

a) The researcher community consists of 20–120 members of the research and teaching staff who are or have been affiliated with the University of Helsinki between 1 January 2005 and 31 October 2010. On 31 October 2010 at least three members of such a group act as Principal Investigators appointed by the University of Helsinki.

b) During the period under evaluation (from 1 January 2005 to 31 December 2010), some members of the researcher community have served as supervisors of doctoral dissertations, appointed for the task by a University of Helsinki faculty.

c) Data on publications and other scientific activities of the researcher community members from 1 January 2005 to 31 December 2010 is updated in the TUHAT database by 31 January 2011.

Moreover, the following conditions were applied to the participating researcher communities:

• Participation is voluntary.

• The participating researcher community may include researchers across department and faculty boundaries.

• The participating researcher communities do not need the approval of their faculty or independent institute, even though it is recommended that faculties and independent institutes encourage their researchers to participate.

• The minimum number of members in a researcher community may be lower for a wellgrounded reason in categories three (research distinct from mainstream research) and four (an innovative opening). However, the members in such a researcher community must include at least two acting Principal Investigators appointed by the University as well as doctoral candidates under their supervision.

• Only Principal Investigators (PI) can participate in the evaluation as members of two researcher communities (A and B). In such cases, the prerequisites are:

• there is only one PI in common between the researcher communities A and B, and

• the researcher communities A and B do not go under the minimum size set for researcher

• communities, and

• the researcher communities A and B participate the evaluation in different categories.

• The participating researcher communities may also include researchers from outside the University of Helsinki. Such researchers will not, however, be included in the number of researchers in the participating researcher community.

Viittaukset

LIITTYVÄT TIEDOSTOT

Justified estimate of the quality of the RC's research and doctoral training at national and international level during 2005-2010 (MAX. 2200 characters with spaces): The results by

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of LMPS – Logic, methodology, and philosophy of

Justified estimate of the quality of the RC's research and doctoral training at national and international level during 2005-2010 (MAX. 2200 characters with spaces): The

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of VARIENG – Research Unit for the Study of

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of MNRP – Research Program of Molecular Neurology.. Type

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of KUFE – Cultural and Feminist Studies in Education..

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of CellMolBiol – The Research Program in Cell and

International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of MATENA – Materials- and Nanophysics