Five Evaluation Panels consisting of independent, renowned and highly respected experts evaluated the publications, scientific activities and other evaluation material of the RCs.
The main domains of the Panels were:
• Biological, Agricultural and Veterinary Sciences
• Medicine, Biomedicine and Health Sciences
• Natural Sciences
• Humanities
• Social Sciences
The University invited 10 renowned scientists to act as Chairs or Vice-Chairs of the Panels based on the suggestions of faculties and independent institutes. Besides leading the work of the Panel, an additional role of the Chairs was to discuss with other Panel Chairs in order to adopt a broadly similar approach.
EVALUATION OF RESEARCH AND DOCTORAL TRAINING 2005–2010
37
2 Implementation of External Evaluation
2.1 Five evaluation Panels
Five Evaluation Panels consisting of independent, renowned and highly respected experts evaluated the publications, scientific activities and other evaluation material of the RCs. The main domains of the Panels were:
1. Biological, Agricultural and Veterinary Sciences 2. Medicine, Biomedicine and Health Sciences 3. Natural Sciences
4. Humanities 5. Social Sciences
The University invited 10 renowned scientists to act as Chairs or Vice-Chairs of the Panels based on the suggestions of faculties and independent institutes. Besides leading the work of the Panel, an additional role of the Chairs was to discuss with other Panel Chairs in order to adopt a broadly similar approach.
The Panel compositions were nominated by the Rector of the University 27th April 2011. The participating RCs suggested the Panel members. The total number of nominated Panel members was 49. In addition to the nominated Panels two external experts participated in their special area in writing feedback. The reason for a smaller number (30%) of panellists as compared to the previous evaluations was the character of the evaluation – meta-evaluation.
The panellists did not read research reports or abstracts but instead, they evaluated answers to the evaluation questions, tables and compilations of publications, other scientific activities and bibliometric or comparable analyses.
0
13 Countries of residence of the panellists
Add. exp.
Panellists
E u r o p e O u t s i d e
E u r o p e
The Panel compositions were nominated by the Rector of the University 27 April 2011.
The participating RCs suggested the Panel members. The total number of nominated Panel members was 49. In addition to the nominated Panels two external experts participated in their special area in writing feedback. The reason for a smaller number (30%) of panellists as compared to the previous evaluations was the character of the evaluation – meta-evaluation.
The panellists did not read research reports or abstracts but instead, they evaluated answers
to the evaluation questions, tables and compilations of publications, other scientific activities and bibliometric or comparable analyses.
The Panels were primarily formed in accordance with divisions between the main fields of science. The Evaluation Steering Group reserved the right to also form Panels representing a narrower selection of fields. The final number of Panels and divisions between fields of research were determined on the basis of field-specific interest in the evaluation. For the evaluation of cross/inter/multidisciplinary research, the RCs were requested to propose a primary field with the most significance from the point of view of research.
The evaluation material was submitted to the Panels in June 2011. Site visits were not organised because interviews for 136 RCs was not possible in practice in the frame of time schedule.
The Panel meetings were held in Helsinki:
• On 11–13 September 2011: (1) Biological, Agricultural and Veterinary Sciences, (2) Medicine, Biomedicine and Health Sciences and (3) Natural Sciences.
• On 18–20 September 2011: (4) Humanities and (5) Social Sciences.
2.2 EVALUATION RELATED TASKS BEFORE THE PANEL MEETINGS IN HELSINKI
The Panel Chairs and Vice-Chairs had a pre-meeting on 27 May 2011 in Amsterdam. The Evaluation Office made a proposal of division of RCs’ material between the panellists. The suggestion was preliminary and the Chairs and Vice-Chairs confirmed the division, made necessary changes together with the evaluation office and completed the division of RCs to the panellists within a week after the Chair meeting in Amsterdam.
Desk-work before the Panel meetings in September 2011
During the time allotted for desk work, each panellist operated both as a first and a second reviewer. In some cases there were more reviewers as well. Each Panel had a detailed reading list that was allocated to pairs of panellists. The pairs of panellists mutually agreed on a working schedule and made sure that the desk work feedback was returned by the deadline of 22 August 2011. The Evaluation Office was informed of any exceptions to the agreed schedule. The desk work was expected to happen mainly in August, but was allowed to be done as soon as all the evaluation material was available in late June.
The exact timing of individual desk work depended on the schedule agreed on between the pairs of reviewers. The first reviewer wrote the initial feedback, and the second reviewer continued writing, disagreed, agreed or added comments. The second reviewer returned the document to the first reviewer. The point was that the preliminary feedback should form a complete draft before the Panel’s meeting in Helsinki. From the point of view of the writing process, it was important that the feedback included argumentation and justification so that it could easily be understood by the other Panel members. Diversity and opposing points were acceptable in the feedback. There were
136 RCs participating in the evaluation, thus each panellist prepared the draft reports for approximately 6–12 RCs. In some cases reviewers evaluated over the Panels.EVALUATION OF RESEARCH AND DOCTORAL TRAINING 2005–2010
39
Figure 1. Example of the pre-work in August and the work in Helsinki in September
The evaluation feedback was to be written in a format similar to that prepared by the Evaluation Office. All the feedback was stored on the Panels’ non-public WIKI site after the first reviewer had sent it to the evaluation office.
Division of labour in the Panel meetings in Helsinki
In September, each Panel met for three days to continue their reporting responsibilities and to finalise the preliminary evaluation feedback. During the meetings in Helsinki, each Panel member was responsible for writing his/her contribution to the preliminary RC-specific reports, which were compiled in the Panel meetings and, when necessary, finalised after the meetings in Helsinki. The Chairs and Vice-Chairs had the main responsibility in the compilation of the feedback. The division of labour in the Panel meetings was decided by the Chairs. Suggestions for practical working methods were introduced to the Panels before the meetings in Helsinki. The Panel worked, in addition to working in sub-groups, as an entity and discussed the questions/aspects/feedback that they considered important to share together in order to have a common understanding about the content matters, recommendations and the numeric level of the feedback. The Panels, however, had the freedom to be flexible in their working methods, and the ways in which those methods were implemented were diverse. Most of the Panels did not work in sub-groups.
The Panel Chairs introduced the preliminary feedback and findings at a general level in the September meetings to the RCs on Tuesday afternoon of the 13thor 20thof September, to which all the RCs’ representatives were invited.
2.3 Reporting aims of the Panels
The Panels gave their feedback at the following levels:
Figure 1. Example of the pre-work in August and the work in Helsinki in September
The evaluation feedback was to be written in a format similar to that prepared by the Evaluation Office. All the feedback was stored on the Panels’ non-public WIKI site after the first reviewer had sent it to the evaluation office.
Division of labour in the Panel meetings in Helsinki
In September, each Panel met for three days to continue their reporting responsibilities and to finalise the preliminary evaluation feedback. During the meetings in Helsinki, each Panel member was responsible for writing his/her contribution to the preliminary RC-specific reports, which were compiled in the Panel meetings and, when necessary, finalised after the meetings in Helsinki. The Chairs and Vice-Chairs had the main responsibility in the compilation of the feedback. The division of labour in the Panel meetings was decided by the Chairs. Suggestions for practical working methods were introduced to the Panels before the meetings in Helsinki. The Panel worked, in addition to working in sub-groups, as an entity and discussed the questions/aspects/feedback that they considered important to share together in order to have a common understanding about the content matters, recommendations and the numeric level of the feedback. The Panels, however, had the freedom to be flexible in their working methods, and the ways in which those methods were implemented were diverse. Most of the Panels did not work in sub-groups.
The Panel Chairs introduced the preliminary feedback and findings at a general level in the September meetings to the RCs on Tuesday afternoon of the 13 or 20 of September, to which all the RCs’ representatives were invited.
2.3 REPORTING AIMS OF THE PANELS
The Panels gave their feedback at the following levels:
TYPE OF REPORT VERSIONS MAIN RESPONSIBILITY
1. RC-specific evaluation feedback 136 separate reports – electronic versions
Panel Members / Chairs 2. University level report in the form of
panel-specific evaluation feedback
one report – paperback and electronic versions
Chairs / Panel Members
The Evaluation Panels were responsible for writing feedback and comments before, during and after the Panel meetings in Helsinki. Most of the written work was to be done in pairs before the Panel meetings. During the process, the Evaluation Office had an option to request for additional argumentation or clarifications from Panel members.
After having received all the feedback the final versions were compiled by the Evaluation Office.
The University level Panel feedback was introduced according to the general themes.
The themes were not common in all the Panels. The final feedback was compiled by the Panel Chairs and Vice-Chairs and was published as such.
2.4 RC-SPECIFIC EVALUATION REPORTS
specific feedback was published electronically as 136 separate reports. The RC-specific reports include mainly the titles as outlined:
• Introduction to the evaluation
• Evaluation feedback to the evaluation questions - considered in parallel with the self-evaluation questions
• Conclusions or summary
• Appendices
- Self-evaluation material
° Registration material
° Answers to evaluation questions - List of participants
- List of publications
- List of other scientific activities - Bibliometric analyses
The panels were encouraged to write honest feedback including thorough arguments and evidence. It was requested that the feedback be evaluative and reflective in addition to any necessary factual descriptions. Evaluative writing was defined as stating what is valuable and describing the relevant background and context.
The panels were expected not to follow the normal distribution in scores but to compare international level in each field of science. In addition to performance, processes were evaluated, as well.
The other important aspect was that the scores should not directly depend on bibliometric indicators but instead to take into account the evaluation material as a whole. The panels stated in detail in RC-specific reports the basis of scoring. The level of scores can be judged only in the RC-specific reports. General impression can be judged reading both RC-specific and university- level reports.
The RC-specific reports were distributed to the RCs by the end of January 2012 and published at the same time as the entire university report, in May 2012.
2.5 UNIVERSITY LEVEL REPORT
The panel specific reports constitute the main part of the University level report. They are presented on a general level and are summaries of the discussions in the panels. The recommendations are addressed generally to the RCs, to the entire university and, in some cases, to the departments and faculties.
The university level report includes five main sections:
1. Introduction to the evaluation – the starting points of the evaluation
2. Panel-specific feedback based on the evaluation material of the Researcher Communities for the panels – five main chapters
Panels have written their feedback based on the RC-specific reports, background documents, bibliometric reports and discussions in the meetings in Helsinki.
Evaluation Office prepared a combination of bibliometrics based on both the CWTS/
Leiden and HULibrary indicators.
3. University level bibliometric report provided by the CWTS/Leiden
The planning of the analyses was done together with the Evaluation Office of the University. The report presents the analyses of the fields of sciences according to the classification of the Web of Science (35 fields). Thus the analyses do not follow the faculty structures or the division of the fields of sciences where the Researcher Communities actually publish their research results. The data is based mainly on the updated publications of RCs not the entire University. Therefore university level conclusions of the performance should be only carefully drawn.
Each Researcher Community, except in the Panels of Humanities and Social Sciences, received detailed bibliometric analysis based on their WoS publications. These analyses have been published only in RC-specific reports.
4. University level bibliometric analyses provided by the Helsinki University Library The Library bibliometric analyses were implemented for the first time at the University
and were experienced very useful. Analyses produced new information about the publishing practices at the University. International, comparative field-normalised indicators are not available and thus the direct comparisons should not be done. Even inside the panels’ fields of sciences the results should be only carefully compared and the conclusions should not be done before the more thorough analyses. The analyses follow the faculty structures and tell about the publishing practices on the faculty level.
RC-specific analyses, mainly for Humanities, Social Sciences and Computer Sciences, were done and published only in RC-specific reports. They shed light on the publishing practices and profiles of the RCs.
5. Doctoral survey carried out by the Centre for Research and Development of Higher Education, the University of Helsinki
The planning and implementation were carried out together with the steering group of the evaluation. The report of doctoral survey follows the faculty structures. The survey provided background material for the panels.
The Panel’s conclusions about the strengths and areas in need of development on the university level were compiled into a report to be published both as a paperback and an electronic version in May 2012.
4 PANEL-SPECIFIC FEEDBACK − MEDICINE, BIOMEDICINE AND HEALTH SCIENCES
5 PANEL-SPECIFIC FEEDBACK − NATURAL SCIENCES
6 PANEL-SPECIFIC FEEDBACK − HUMANITIES
7 PANEL-SPECIFIC FEEDBACK − SOCIAL SCIENCES
8 OVERALL STATISTICS ON THE EVALUATION
9 RESEARCH PERFORMANCE
10 BIBLIOMETRIC ANALYSES
11 SUMMARY REPORT OF DOCTORAL TRAINING
12 CONCLUDING REMARKS
13 APPENDICES 2 IMPLEMENTATION OF THE EXTERNAL EVALUATION
3 PANEL-SPECIFIC FEEDBACK − BIOLOGICAL, AGRICULTURAL AND VETERINARY SCIENCES
PANEL-SPECIFIC FEEDBACK − BIOLOGICAL, AGRICULTURAL AND VETERINARY SCIENCES
Panel members
Professor Ary A. Hoffman, Chair Professor Barbara Koch, Vice-Chair Professor Per-Anders Hansson Professor Danny Huylebroeck Professor Jonathan King Professor Hannu J.T. Korhonen Professor Kristiina Kruus Professor Joakim Lundeberg Professor Dominiek Maes Professor Olli Saastamoinen Professor Kai Simons
Researcher Communities in the Panel of Biological, Agricultural and Veterinary Sciences
Number of RCs 27, principal investigators 249 and other members 1,096
ACRONYM
OF RC NAME OF RC RESPONSIBLE
PERSON OF RC PIS MBS. TOT.
ARC Avian Research Community Brommer, Jon 5 22 27
BIOSYST Biological Systematics and Taxonomy Stenroos, Soili 13 25 38 CellMolBiol The Research Program in Cell and Molecular
Biology
Lappalainen, Pekka 7 40 47 CoE MRG Centre of Excellence in Metapopulation Research Hanski, Ilkka 8 47 55 CoE_VIRRES Center of Excellence in Virus Research Bamford, Dennis 6 37 43
CoE-MiFoSaPLUS
Extended Center of Excellence in Microbiology and Food Safety Research
Palva, Airi 13 77 90
EGRU Ecological Genetics Research Unit Merilä, Juha 5 25 30
ENIGMA Environmental change and management Korhola, Atte 9 24 33
EvoDevo Helsinki EvoDevo Jernvall, Jukka 5 16 21
FoodNutri Food and Nutrition Sciences Lamberg-Allardt, Christel
20 78 98
FRESH Freshwater Research Horppila, Jukka 8 20 28
HelDevBio Helsinki Developmental Biology Research Community
Thesleff, Irma 11 54 65
INBIOS Integrative Biodiversity Science Rikkinen, Jouko 11 30 41
LEGMILK Home-grown feeds, milk and healthy aging Wähälä, Kristiina 4 18 22 Membrec Cell membrane recognition and dynamics Gahmberg, Carl G. 7 36 43 MICRO Research and postgraduate training in
microbiology
Sivonen, Kaarina 10 75 85
MUSGEN Gene-culture evolution in music Järvelä, Irma 2 8 10
PEATLANDERS Peatland Ecology Group in the University of Helsinki
Vasander, Harri 9 22 31
PHABIO Pharmaceutical Biology Vuorela, Heikki 3 22 25
PHYTOPATH Phytopathogen Research Valkonen, Jari 4 43 47
SB&B Structural Biology & Biophysics Programme Wikström, Mårten 10 29 39 SSA Science of Sustainable Agriculture Stoddard,
Frederick
27 68 95
SUVALUE Sustainable Forest Value Chains Valsta, Lauri 11 48 59
VetSci Veterinary science: clinical, translational, and animal welfare research
Peltoniemi, Olli 19 93 112 ViiGen Viikki Genome Biology Research Community Helariutta, Yrjö 7 35 42 VITRI Viikki Tropical Resources Institute Luukkanen, Olavi 3 30 33
VMPS Viikki Molecular Plant Sciences Palva, Tapio 12 74 86
The panel’s feedback is presented in chapters 3, 3.1–3.2. The tables and figures in chapter 3 are compiled by the Evaluation Office based on the statistics of CWTS/Leiden and the Helsinki University Library.
Fields of sciences of the Researcher Communities in the Panel of Biological, Agricultural and Veterinary Sciences
RC SUBFIELD 1 SUBFIELD 2 SUBFIELD 3 SUBFIELD 4 OTHER SCIENTIFIC SUBFIELD ARC Evolutionary
Biology
Ecology Ornithology Genetics and Heredity
BIOSYST Biology Mycology Plant Sciences Zoology Systematics and Taxonomy CellMolBiol Cell Biology Biochemistry
and Molecular Biology
-
-CoE MRG Ecology Evolutionary Biology
Microscopy Microbiology Bioinformatics Structural biology
Ecology Zoology Biodiversity Conservation, Fisheries, Marine and
-FRESH Limnology Marine and Freshwater
Molecular chemistry in life science Organic analysis
-RC SUBFIELD 1 SUBFIELD 2 SUBFIELD 3 SUBFIELD 4 OTHER SCIENTIFIC
Peatlanders Ecology Forestry Biochemistry and Molecular
-SUVALUE Forestry Economics Operations Research and Management Science
Remote Sensing Techonology and Engineering; Materials
-VMPS Plant Sciences Biochemistry and Molecular Biology
Cell Biology Developmental Biology
Genetics and genomics
The table shows the fields of sciences chosen by the RCs in the Panel of Biological, Agricultural and Veterinary Sciences. Subfields 1−4 follow the classification of Web of Science (for a complete list, see Appendix 4), and “other scientific subfield” is the RC’s own description.
3. PANEL-SPECIFIC FEEDBACK − BIOLOGICAL, AGRICULTURAL AND VETERINARY SCIENCES
The Panel was positive about the RC concept. RCs provide flexible units that have the potential to capture research units without the constraint of departmental boundaries. By facilitating the establishment of RCs, The University of Helsinki has the potential to develop dynamic research units that can explore multi- and inter-disciplinary research and capture expertise available within the University. These units may then mature into groups that bid for centre of excellence funds and/or funds from other organizations including those available through EC schemes. By building a profile within the university and the wider community, RCs have the potential to have an impact at the societal-level and to be widely recognized for their expertise. They may also form effective units for acquiring and effectively using large items of equipment and other infrastructure without aiming themselves for becoming large infrastructure facilities only, which should be funded separately.
The BIO sector at UH is in a healthy position, and includes several world class research groups as well as nationally significant training centres feeding into
The BIO sector at UH is in a healthy position, and includes several world class research groups as well as nationally significant training centres feeding into