Evaluation Panel: Biological, Agricultural and Veterinary Sciences INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL
TRAINING AT THE UNIVERSITY OF HELSINKI 2005–2010
RC-Specific Evaluation of CellMolBiol – The Research Program in Cell and Molecular Biology
Seppo Saari & Antti Moilanen (Eds.)
University of Helsinki
Administrative Publications 80/11 Evaluations
INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL TRAINING AT THE UNIVERSITY OF HELSINKI 2005–2010
RC-Specific Evaluation of CellMolBiol – The Research Program in Cell and Molecular Biology
Seppo Saari & Antti Moilanen (Eds.)
2012
Publisher:
University of Helsinki Editors:
Seppo Saari & Antti Moilanen
Title:
International Evaluation of Research and Doctoral Training at the University of Helsinki 2005–2010 : RC-Specific Evaluation of CellMolBiol – The Research Program in Cell and Molecular Biology
Type of publication:
Evaluations
Summary:
Researcher Community (RC) was a new concept of the participating unit in the evaluation. Participation in the evaluation was voluntary and the RCs had to choose one of the five characteristic categories to participate.
Evaluation of the Researcher Community was based on the answers to the evaluation questions. In addition a list of publications and other activities were provided by the TUHAT system. The CWTS/Leiden University conducted analyses for 80 RCs and the Helsinki University Library for 66 RCs.
Panellists, 49 and two special experts in five panels evaluated all the evaluation material as a whole and discussed the feedback for RC-specific reports in the panel meetings in Helsinki. The main part of this report is consisted of the feedback which is published as such in the report.
Chapters in the report:
1. Background for the evaluation
2. Evaluation feedback for the Researcher Community 3. List of publications
4. List of activities 5. Bibliometric analyses
The level of the RCs’ success can be concluded from the written feedback together with the numeric evaluation of four evaluation questions and the category fitness. More conclusions of the success can be drawn based on the University-level report.
RC-specific information:
Main scientific field of research:
Biological, Agricultural and Veterinary Sciences
Participation category:
1. Research of the participating community represents the international cutting edge in its field
RC’s responsible person:
Lappalainen, Pekka
RC-specific keywords:
Cell biology, molecular biology, cytoskeleton,
transcription, cell growth, endoplasmic reticulum, virus, neuron, secretion, microscopy
Keywords:
Research Evaluation, Meta-evaluation, Doctoral Training, Bibliometric Analyses, Researcher Community
Series title and number:
University of Helsinki, Administrative Publications 80/11, Evaluations ISSN:
1795-5513 (Online)
ISBN:
978-952-10-7431-8 (PDF) Total number of pages:
63
Language:
English Additional information:
Cover graphics: Päivi Talonpoika-Ukkonen Enquiries: seppo.o.saari@helsinki.fi
Internet address:
http://www.helsinki.fi/julkaisut/aineisto/rc_evaluation 2012/hallinnon_julkaisuja_80_11_2012.pdf
Contents
Panel members ... 1
1 Introduction to the Evaluation ... 5
1.1 RC-specific evaluation reports ... 5
1.2 Aims and objectives in the evaluation ... 5
1.3 Evaluation method ... 5
1.4 Implementation of the external evaluation ... 6
1.5 Evaluation material ... 7
1.6 Evaluation questions and material ... 8
1.7 Evaluation criteria ... 10
1.8 Timetable of the evaluation ... 13
1.9 Evaluation feedback – consensus of the entire panel ... 13
2 Evaluation feedback ... 15
2.1 Focus and quality of the RC’s research ... 15
2.2 Practises and quality of doctoral training ... 15
2.3 The societal impact of research and doctoral training ... 16
2.4 International and national (incl. intersectoral) research collaboration and researcher mobility ... 16
2.5 Operational conditions ... 16
2.6 Leadership and management in the researcher community ... 16
2.7 External competitive funding of the RC ... 17
2.8 The RC’s strategic action plan for 2011–2013 ... 17
2.9 Evaluation of the category of the RC in the context of entity of the evaluation material (1-8) ... 17
2.10 Short description of how the RC members contributed the compilation of the stage 2 material ... 18
2.11 How the UH’s focus areas are presented in the RC’s research ... 18
2.12 RC-specific main recommendations ... 18
2.13 RC-specific conclusions ... 18
3 Appendices ... 19
Foreword
The evaluation of research and doctoral training is being carried out in the years 2010–2012 and will end in 2012. The steering group appointed by the Rector in January 2010 set the conditions for participating in the evaluation and prepared the Terms of Reference to present the evaluation procedure and criteria. The publications and other scientific activities included in the evaluation covered the years 2005–2010.
The participating unit in the evaluation was defined as a Researcher Community (RC). To obtain a critical mass with university-level impact, the number of members was set to range from 20 to 120. The RCs were required to contain researchers in all stages of their research career, from doctoral students to principal investigators (PIs). All in all, 136 Researcher Communities participated in this voluntary evaluation, 5857 persons in total, of whom 1131 were principal investigators. PIs were allowed to participate in two communities in certain cases, and 72 of them used this opportunity and participated in two RCs.
This evaluation enabled researchers to define RCs from the “bottom up” and across disciplines. The aim of the evaluation was not to assess individual performance but a community with shared aims and researcher-training activities. The RCs were able to choose among five different categories that characterised the status and main aims of their research. The steering group considered the process of applying to participate in the evaluation to be important, which lead to the establishment of these categories. In addition, providing a service for the RCs to enable them to benchmark their research at the global level was a main goal of the evaluation.
The data for the evaluation consisted of the RCs’ answers to evaluation questions on supplied e-forms and a compilation extracted from the TUHAT – Research Information System (RIS) on 12 April 2011. The compilation covered scientific and other publications as well as certain areas of scientific activities. During the process, the RCs were asked to check the list of publications and other scientific activities and make corrections if needed. These TUHAT compilations are public and available on the evaluation project sites of each RC in the TUHAT-RIS.
In addition to the e-form and TUHAT compilation, University of Leiden (CWTS) carried out bibliometric analyses from the articles included in the Web of Science (WoS). This was done on University and RC levels. In cases where the publication forums of the RC were clearly not represented by the WoS data, the Library of the University of Helsinki conducted a separate analysis of the publications. This was done for 66 RCs representing the humanities and social sciences.
The evaluation office also carried out an enquiry targeted to the supervisors and PhD candidates about the organisation of doctoral studies at the University of Helsinki. This and other documents describing the University and the Finnish higher education system were provided to the panellists.
The panel feedback for each RC is unique and presented as an entity. The first collective evaluation reports available for the whole panel were prepared in July–August 2011. The reports were accessible to all panel members via the electronic evaluation platform in August. Scoring from 1 to 5 was used to complement written feedback in association with evaluation questions 1–4 (scientific focus and quality, doctoral training, societal impact, cooperation) and in addition to the category evaluating the fitness for participation in the evaluation. Panellists used the international level as a point of comparison in the evaluation. Scoring was not expected to go along with a preset deviation.
Each of the draft reports were discussed and dealt with by the panel in meetings in Helsinki (from 11 September to 13 September or from 18 September to 20 September 2011). In these meetings the panels also examined the deviations among the scores and finalised the draft reports together.
The current RC-specific report deals shortly with the background of the evaluation and the terms of participation. The main evaluation feedback is provided in the evaluation report, organised according to the evaluation questions. The original material provided by the RCs for the panellists has been attached to these documents.
On behalf of the evaluation steering group and office, I sincerely wish to thank you warmly for your participation in this evaluation. The effort you made in submitting the data to TUHAT-RIS is gratefully acknowledged by the University. We wish that you find this panel feedback useful in many ways. The bibliometric profiles may open a new view on your publication forums and provide a perspective for discussion on your choice of forums. We especially hope that this evaluation report will help you in setting the future goals of your research.
Johanna Björkroth Vice-Rector
Chair of the Steering Group of the Evaluation
Steering Group of the evaluation
Steering group, nominated by the Rector of the University, was responsible for the planning of the evaluation and its implementation having altogether 22 meetings between February 2010 and March 2012.
Chair
Vice-Rector, professor Johanna Björkroth Vice-Chair
Professor Marja Airaksinen
Chief Information Specialist, Dr Maria Forsman Professor Arto Mustajoki
University Lecturer, Dr Kirsi Pyhältö
Director of Strategic Planning and Development, Dr Ossi Tuomi Doctoral candidate, MSocSc Jussi Vauhkonen
1
Panel members
CHAIR
Professor Ary A. Hoffman
Ecological genetics, evolutionary biology, biodiversity conservation, zoology University of Melbourne, Australia VICE-CHAIR
Professor Barbara Koch Forest Sciences, remote sensing University of Freiburg, Germany Professor Per-Anders Hansson
Agricultural engineering, modeling, life cycle analysis, bioenergy
Swedish University of Agricultural Sciences Professor Danny Huylebroeck
Developmental biology
Katholieke Universiteit Leuven, Belgium Professor Jonathan King
Virus assembly, protein folding
Massachusetts Institute of Technology MIT, USA Professor Hannu J.T. Korhonen
Functional foods, dairy technology, milk hygiene MTT Agrifood Research Finland
Professor Kristiina Kruus
Microbiological biotechnology, microbiological enzymes, applied microbiology
VTT Technical Research Centre of Finland Professor Joakim Lundeberg
Biochemistry, biotechnology, sequencing, genomics KTH Royal Institute of Technology, Sweden
Professor Dominiek Maes Veterinary medicine Ghent University, Belgium Professor Olli Saastamoinen Forest economics and policy University of Eastern Finland Professor Kai Simons
Biochemistry, molecular biology, cell biology Max-Planck-Institute of Molecular Cell Biology and Genetics, Germany
The panel, independently, evaluated all the submitted material and was responsible for the feedback of the RC-specific reports. The panel members were asked to confirm whether they had any conflict of interests with the RCs. If this was the case, the panel members disqualified themselves in discussion and report writing.
Added expertise to the evaluation was contributed by the members from the other panels and by one evaluator outside the panels.
2
External Expert Professor Anders Linde Oral biochemi
Faculty of Odontology Göteborg University Sweden
Experts from the Other Panels
Professor Caitlin Buck, from the Panel of Natural Sciences Professor Ritske Huismans, from the Panel of Natural Sciences
Professor Johanna Ivaska, from the Panel of Medicine, biomedicine and health sciences Professor Lea Kauppi, from the Panel of Natural Sciences
Professor Holger Stark, from the Panel of Natural Sciences
Professor Peter York, from the Panel of Medicine, biomedicine and health sciences EVALUATION OFFICE
Dr Seppo Saari, Doc., Senior Adviser in Evaluation, was responsible for the entire evaluation, its planning and implementation and acted as an Editor-in-chief of the reports.
Dr Eeva Sievi, Doc., Adviser, was responsible for the registration and evaluation material compilations for the panellists. She worked in the evaluation office from August 2010 to July 2011.
MSocSc Paula Ranne, Planning Officer, was responsible for organising the panel meetings and all the other practical issues like agreements and fees and editing a part the RC-specific reports. She worked in the evaluation office from March 2011 to January 2012.
Mr Antti Moilanen, Project Secretary, was responsible for editing the reports. He worked in the evaluation office from January 2012 to April 2012.
TUHAT OFFICE
Provision of the publication and other scientific activity data
Mrs Aija Kaitera, Project Manager of TUHAT-RIS served the project ex officio providing the evaluation project with the updated information from TUHAT-RIS.
The TUHAT office assisted in mapping the publications with CWTS/University of Leiden.
MA Liisa Ekebom, Assisting Officer, served in TUHAT-RIS updating the publications for the evaluation. She also assisted the UH/Library analyses.
BA Liisa Jäppinen, Assisting Officer, served in TUHAT-RIS updating the publications for the evaluation.
HELSINKI UNIVERSITY LIBRARY Provision of the publication analyses
Dr Maria Forsman, Chief Information Specialist in the Helsinki University Library, managed with her 10 colleagues the bibliometric analyses in humanities, social sciences and in other fields of sciences where CWTS analyses were not applicable.
3 Acronyms and abbreviations applied in the report
External competitive funding AF – Academy of Finland
TEKES - Finnish Funding Agency for Technology and Innovation EU - European Union
ERC - European Research Council International and national foundations
FP7/6 etc. /Framework Programmes/Funding of European Commission Evaluation marks
Outstanding (5) Excellent (4) Very Good (3) Good (2) Sufficient (1)
Abbreviations of Bibliometric Indicators P - Number of publications
TCS – Total number of citations
MCS - Number of citations per publication, excluding self-citations PNC - Percentage of uncited publications
MNCS - Field-normalized number of citations per publication MNJS - Field-normalized average journal impact
THCP10 - Field-normalized proportion highly cited publications (top 10%)
INT_COV - Internal coverage, the average amount of references covered by the WoS WoS – Thomson Reuters Web of Science Databases
Participation category
Category 1. The research of the participating community represents the international cutting edge in its field.
Category 2. The research of the participating community is of high quality, but the community in its present composition has yet to achieve strong international recognition or a clear break-through.
Category 3. The research of the participating community is distinct from mainstream research, and the special features of the research tradition in the field must be considered in the evaluation.
Category 4. The research of the participating community represents an innovative opening.
Category 5. The research of the participating community has a highly significant societal impact.
Research focus areas of the University of Helsinki
Focus area 1: The basic structure, materials and natural resources of the physical world Focus area 2: The basic structure of life
Focus area 3: The changing environment – clean water Focus area 4: The thinking and learning human being Focus area 5: Welfare and safety
Focus area 6: Clinical research Focus area 7: Precise reasoning Focus area 8: Language and culture Focus area 9: Social justice
Focus area 10: Globalisation and social change
4
5
1 Introduction to the Evaluation
1.1 RC-specific evaluation reports
The participants in the evaluation of research and doctoral training were Researcher Communities (hereafter referred to as the RC). The RC refers to the group of researchers who registered together in the evaluation of their research and doctoral training. Preconditions in forming RCs were stated in the Guidelines for the Participating Researcher Communities. The RCs defined themselves whether their compositions should be considered well-established or new.
It is essential to emphasise that the evaluation combines both meta-evaluation1 and traditional research assessment exercise and its focus is both on the research outcomes and procedures associated with research and doctoral training. The approach to the evaluation is enhancement-led where self- evaluation constituted the main information. The answers to the evaluation questions formed together with the information of publications and other scientific activities an entity that was to be reviewed as a whole.
The present evaluation recognizes and justifies the diversity of research practices and publication traditions. Traditional Research Assessment Exercises do not necessarily value high quality research with low volumes or research distinct from mainstream research. It is challenging to expose the diversity of research to fair comparison. To understand the essence of different research practices and to do justice to their diversity was one of the main challenges of the present evaluation method. Understanding the divergent starting points of the RCs demanded sensitivity from the evaluators.
1.2 Aims and objectives in the evaluation
The aims of the evaluation are as follows:
to improve the level of research and doctoral training at the University of Helsinki and to raise their international profile in accordance with the University’s strategic policies. The improvement of doctoral training should be compared to the University’s policy.2
to enhance the research conducted at the University by taking into account the diversity, originality, multidisciplinary nature, success and field-specificity,
to recognize the conditions and prerequisites under which excellent, original and high-impact research is carried out,
to offer the academic community the opportunity to receive topical and versatile international peer feedback,
to better recognize the University’s research potential.
to exploit the University’s TUHAT research information system to enable transparency of publishing activities and in the production of reliable, comparable data.
1.3 Evaluation method
The evaluation can be considered as an enhancement-led evaluation. Instead of ranking, the main aim is to provide useful information for the enhancement of research and doctoral training of the participating RCs.
The comparison should take into account each field of science and acknowledge their special character.
1 The panellists did not read research reports or abstracts but instead, they evaluated answers to the evaluation questions, tables and compilations of publications, other scientific activities, bibliometrics or comparable analyses.
2 Policies on doctoral degrees and other postgraduate degrees at the University of Helsinki.
6
The comparison produced information about the present status and factors that have lead to success. Also challenges in the operations and outcomes were recognized.
The evaluation approach has been designed to recognize better the significance and specific nature of researcher communities and research areas in the multidisciplinary top-level university. Furthermore, one of the aims of the evaluation is to bring to light those evaluation aspects that differ from the prevalent ones. Thus the views of various fields of research can be described and research arising from various starting points understood better. The doctoral training is integrated into the evaluation as a natural component related to research. Operational processes of doctoral training are being examined in the evaluation.
Five stages of the evaluation method were:
1. Registration – Stage 1 2. Self-evaluation – Stage 2
3. TUHAT3 compilations on publications and other scientific activities4 4. External evaluation
5. Public reporting
1.4 Implementation of the external evaluation
Five Evaluation Panels
Five evaluation panels consisted of independent, renowned and highly respected experts. The main domains of the panels are:
1. biological, agricultural and veterinary sciences 2. medicine, biomedicine and health sciences 3. natural sciences
4. humanities 5. social sciences
The University invited 10 renowned scientists to act as chairs or vice-chairs of the five panels based on the suggestions of faculties and independent institutes. Besides leading the work of the panel, an additional role of the chairs was to discuss with other panel chairs in order to adopt a broadly similar approach. The panel chairs and vice-chairs had a pre-meeting on 27 May 2011 in Amsterdam.
The panel compositions were nominated by the Rector of the University 27 April 2011. The participating RCs suggested the panel members. The total number of panel members was 50. The reason for a smaller number of panellists as compared to the previous evaluations was the character of the evaluation as a meta-evaluation. The panellists did not read research reports or abstracts but instead, they evaluated answers to the evaluation questions, tables and compilations of publications, other scientific activities, bibliometrics and comparable analyses.
The panel meetings were held in Helsinki:
On 11–13 September 2011: (1) biological, agricultural and veterinary sciences, (2) medicine, biomedicine and health sciences and (3) natural sciences.
On 18–20 September 2011: (4) humanities and (5) social sciences.
3 TUHAT (acronym) of Research Information System (RIS) of the University of Helsinki
4 Supervision of thesis, prizes and awards, editorial work and peer reviews, participation in committees, boards and networks and public appearances.
7
1.5 Evaluation material
The main material in the evaluation was the RCs’ self-evaluations that were qualitative in character and allowed the RCs to choose what was important to mention or emphasise and what was left unmentioned.
The present evaluation is exceptional at least in the Finnish context because it is based on both the evaluation documentation (self-evaluation questions, publications and other scientific activities) and the bibliometric reports. All documents were delivered to the panellists for examination.
Traditional bibliometrics can be reasonably done mainly in medicine, biosciences and natural sciences when using the Web of Science database, for example. Bibliometrics, provided by CWTS/The Centre for Science and Technology Studies, University of Leiden, cover only the publications that include WoS identification in the TUHAT-RIS.
Traditional bibliometrics are seldom relevant in humanities and social sciences because the international comparable databases do not store every type of high quality research publications, such as books and monographs and scientific journals in other languages than English. The Helsinki University Library has done analysis to the RCs, if their publications were not well represented in the Web of Science databases (RCs should have at least 50 publications and internal coverage of publications more than 40%) – it meant 58 RCs. The bibliometric material for the evaluation panels was available in June 2011. The RC- specific bibliometric reports are attached at the end of each report.
The panels were provided with the evaluation material and all other necessary background information, such as the basic information about the University of Helsinki and the Finnish higher education system.
Evaluation material
1. Registration documents of the RCs for the background information 2. Self evaluation material – answers to the evaluation questions 3. Publications and other scientific activities based on the TUHAT RIS:
3.1. statistics of publications 3.2. list of publications
3.3. statistics of other scientific activities 3.4. list of other scientific activities 4. Bibliometrics and comparable analyses:
4.1. Analyses of publications based on the verification of TUHAT-RIS publications with the Web of Science publications (CWTS/University of Leiden)
4.2. Publication statistics analysed by the Helsinki University Library - mainly for humanities and social sciences
5. University level survey on doctoral training (August 2011)
6. University level analysis on publications 2005–2010 (August 2011) provided by CWTS/University of Leiden
Background material University of Helsinki
- Basic information about the University of the Helsinki - The structure of doctoral training at the University of Helsinki
- Previous evaluations of research at the University of Helsinki – links to the reports: 1998 and 2005 The Finnish Universities/Research Institutes
- Finnish University system
- Evaluation of the Finnish National Innovation System
- The State and Quality of Scientific Research in Finland. Publication of the Academy of Finland 9/09.
The evaluation panels were provided also with other relevant material on request before the meetings in Helsinki.
8
1.6 Evaluation questions and material
The participating RCs answered the following evaluation questions which are presented according to the evaluation form. In addition, TUHAT RIS was used to provide the additional material as explained. For giving the feedback to the RCs, the panellists received the evaluation feedback form constructed in line with the evaluation questions:
1. Focus and quality of the RC’s research
Description of
- the RC’s research focus.
- the quality of the RC’s research (incl. key research questions and results) - the scientific significance of the RC’s research in the research field(s)
Identification of the ways to strengthen the focus and improve the quality of the RC’s research The additional material: TUHAT compilation of the RC’s publications, analysis of the RC’s publications data (provided by University of Leiden and the Helsinki University Library)
A written feedback from the aspects of: scientific quality, scientific significance, societal impact, innovativeness
Strengths
Areas of development
Other remarks
Recommendations
Numeric evaluation: OUTSTANDING (5), EXCELLENT (4), VERY GOOD (3), GOOD (2), SUFFICIENT (1) 2. Practises and quality of doctoral training
Organising of the doctoral training in the RC. Description of the RC’s principles for:
- recruitment and selection of doctoral candidates - supervision of doctoral candidates
- collaboration with faculties, departments/institutes, and potential graduate schools/doctoral programmes
- good practises and quality assurance in doctoral training
- assuring of good career perspectives for the doctoral candidates/fresh doctorates
Identification of the RC’s strengths and challenges related to the practises and quality of doctoral training, and the actions planned for their development.
The additional material: TUHAT compilation of the RC’s other scientific activities/supervision of doctoral dissertations
A written feedback from the aspects of: processes and good practices related to leadership and management
Strengths
Areas of development
Other remarks
Recommendations
Numeric evaluation: OUTSTANDING (5), EXCELLENT (4), VERY GOOD (3), GOOD (2), SUFFICIENT (1) 3. The societal impact of research and doctoral training
Description on how the RC interacts with and contributes to the society (collaboration with public, private and/or 3rd sector).
Identification of the ways to strengthen the societal impact of the RC’s research and doctoral training.
The additional material: TUHAT compilation of the RC’s other scientific activities.
A written feedback from the aspects of: societal impact, national and international collaboration, innovativeness
Strengths
Areas of development
Other remarks
Recommendations
Numeric evaluation: OUTSTANDING (5), EXCELLENT (4), VERY GOOD (3), GOOD (2), SUFFICIENT (1)
9 4. International and national (incl. intersectoral) research collaboration and researcher mobility
Description of
- the RC’s research collaborations and joint doctoral training activities - how the RC has promoted researcher mobility
Identification of the RC’s strengths and challenges related to research collaboration and researcher mobility, and the actions planned for their development.
A written feedback from the aspects of: scientific quality, national and international collaboration
Strengths
Areas of development
Other remarks
Recommendations
Numeric evaluation: OUTSTANDING (5), EXCELLENT (4), VERY GOOD (3), GOOD (2), SUFFICIENT (1) 5. Operational conditions
Description of the operational conditions in the RC’s research environment (e.g. research infrastructure, balance between research and teaching duties).
Identification of the RC’s strengths and challenges related to operational conditions, and the actions planned for their development.
A written feedback from the aspects of: processes and good practices related to leadership and management
Strengths
Areas of development
Other remarks
Recommendations
6. Leadership and management in the researcher community
Description of
- the execution and processes of leadership in the RC
- how the management-related responsibilities and roles are distributed in the RC - how the leadership- and management-related processes support
- high quality research
- collaboration between principal investigators and other researchers in the RC the RC’s research focus
- strengthening of the RC’s know-how
Identification of the RC’s strengths and challenges related to leadership and management, and the actions planned for developing the processes
7. External competitive funding of the RC
The RCs were asked to provide information of such external competitive funding, where:
- the funding decisions have been made during 1.1.2005-31.12.2010, and - the administrator of the funding is/has been the University of Helsinki
On the e-form the RCs were asked to provide:
1) The relevant funding source(s) from a given list (Academy of Finland/Research Council, TEKES/The Finnish Funding Agency for Technology and Innovation , EU, ERC, foundations, other national funding organisations, other international funding organisations), and
2)The total sum of funding which the organisation in question had decided to allocate to the RCs members during 1.1.2005–31.12.2010.
Competitive funding reported in the text is also to be considered when evaluating this point.
A written feedback from the aspects of: scientific quality, scientific significance, societal impact, innovativeness, future significance
Strengths
Areas of development
Other remarks
Recommendations
8. The RC’s strategic action plan for 2011–2013
RC’s description of their future perspectives in relation to research and doctoral training.
A written feedback from the aspects of: scientific quality, scientific significance, societal Impact, processes and good practices related to leadership and management, national and international collaboration, innovativeness, future significance
Strengths
Areas of development
10
Other remarks
Recommendations
9. Evaluation of the category of the RC in the context of entity of the evaluation material (1-8) The RC’s fitness to the chosen participation category
A written feedback evaluating the RC’s fitness to the chosen participation category
Strengths
Areas of development
Other remarks
Recommendations
Numeric evaluation: OUTSTANDING (5), EXCELLENT (4), VERY GOOD (3), GOOD (2), SUFFICIENT (1) 10. Short description of how the RC members contributed the compilation of the stage 2 material Comments on the compilation of evaluation material
11. How the UH’s focus areas are presented in the RC’s research?
Comments if applicable
12. RC-specific main recommendations based on the previous questions 1–11 13. RC-specific conclusions
1.7 Evaluation criteria
The panellists were expected to give evaluative and analytical feedback to each evaluation question according to their aspects in order to describe and justify the quality of the submitted material. In addition, the evaluation feedback was asked to be pointed out the level of the performance according to the following classifications:
outstanding (5)
excellent (4)
very good (3)
good (2)
sufficient (1)
Evaluation according to the criteria was to be made with thorough consideration of the entire evaluation material of the RC in question. Finally, in questions 1-4 and 9, the panellists were expected to classify their written feedback into one of the provided levels (the levels included respective descriptions,
‘criteria’). Some panels used decimals in marks. The descriptive level was interpreted according to the integers and not rounding up the decimals by the editors.
Description of criteria levels
Question 1 – FOCUS AND QUALITY OF THE RC’S RESEARCH Classification: Criteria (level of procedures and results) Outstanding quality of procedures and results (5)
Outstandingly strong research, also from international perspective. Attracts great international interest with a wide impact, including publications in leading journals and/or monographs published by leading international publishing houses. The research has world leading qualities. The research focus, key research questions scientific significance, societal impact and innovativeness are of outstanding quality.
In cases where the research is of a national character and, in the judgement of the evaluators, should remain so, the concepts of ”international attention” or ”international impact” etc. in the grading criteria above may be replaced by ”international comparability”.
11 Operations and procedures are of outstanding quality, transparent and shared in the community. The improvement of research and other efforts are documented and operations and practices are in alignment with the documentation. The ambition to develop the community together is of outstanding quality.
Excellent quality of procedures and results (4)
Research of excellent quality. Typically published with great impact, also internationally. Without doubt, the research has a leading position in its field in Finland.
Operations and procedures are of excellent quality, transparent and shared in the community. The improvement of research and other efforts are documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of excellent quality.
Very good quality of procedures and results (3)
The research is of such very good quality that it attracts wide national and international attention.
Operations and procedures are of very good quality, transparent and shared in the community. The improvement of research and other efforts are documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of very good quality.
Good quality of procedures and results (2)
Good research attracting mainly national attention but possessing international potential, extraordinarily high relevance may motivate good research.
Operations and procedures are of good quality, shared occasionally in the community. The improvement of research and other efforts are occasionally documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of good quality.
Sufficient quality of procedures and results (1)
In some cases the research is insufficient and reports do not gain wide circulation or do not have national or international attention. Research activities should be revised.
Operations and procedures are of sufficient quality, shared occasionally in the community. The improvement of research and other efforts are occasionally documented and operations and practices are to some extent in alignment with the documentation. The ambition to develop the community together is of sufficient quality.
Question 2 – DOCTORAL TRAINING Question 3 – SOCIETAL IMPACT Question 4 – COLLABORATION
Classification: Criteria (level of procedures and results) Outstanding quality of procedures and results (5)
Procedures are of outstanding quality, transparent and shared in the community. The practices and quality of doctoral training/societal impact/international and national collaboration/leadership and management are documented and operations and practices are in alignment with the documentation. The ambition to develop the community together is of outstanding quality. The procedures and results are regularly evaluated and the feedback has an effect on the planning.
Excellent quality of procedures and results (4)
Procedures are of excellent quality, transparent and shared in the community. The practices and quality of doctoral training/societal impact/international and national collaboration/leadership and management are documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of excellent quality. The procedures and outcomes are evaluated and the feedback has an effect on the planning.
Very good quality of procedures and results (3)
Procedures are of very good quality, transparent and shared in the community. The practices and quality of doctoral training/societal impact/international and national collaboration/leadership and
12
management are documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of very good quality.
Good quality of procedures and results (2)
Procedures are of good quality, shared occasionally in the community. The practices and quality of doctoral training/societal impact/international and national collaboration/leadership and management are documented and operations and practices are to large extent in alignment with the documentation. The ambition to develop the community together is of good quality.
Sufficient quality of procedures and results (1)
Procedures are of sufficient quality, transparent and shared in the community. The practices and quality of doctoral training/societal impact/international and national collaboration/leadership and management are occasionally documented and operations and practices are to some extent in alignment with the documentation. The ambition to develop the community together is of sufficient quality.
Question 9 – CATEGORY
Participation category – fitness for the category chosen
The choice and justification for the chosen category below should be reflected in the RC’s responses to the evaluation questions 1–8.
1. The research of the participating community represents the international cutting edge in its field.
2. The research of the participating community is of high quality, but the community in its present composition has yet to achieve strong international recognition or a clear break-through.
3. The research of the participating community is distinct from mainstream research, and the special features of the research tradition in the field must be considered in the evaluation. The research is of high quality and has great significance and impact in its field. However, the generally used research evaluation methods do not necessarily shed sufficient light on the merits of the research.
4. The research of the participating community represents an innovative opening. A new opening can be an innovative combination of research fields, or it can be proven to have a special social, national or international demand or other significance. Even if the researcher community in its present composition has yet to obtain proof of international success, its members can produce convincing evidence of the high level of their previous research.
5. The research of the participating community has a highly significant societal impact. The participating researcher community is able to justify the high social significance of its research.
The research may relate to national legislation, media visibility or participation in social debate, or other activities promoting social development and human welfare. In addition to having societal impact, the research must be of a high standard.
An example of outstanding fitness for category choice (5) 5
The RC’s representation and argumentation for the chosen category were convincing. The RC recognized its real capacity and apparent outcomes in a wider context to the research communities. The specific character of the RC was well-recognized and well stated in the responses. The RC fitted optimally for the category.
Outstanding (5)
Excellent (4)
Very good (3)
Good (2)
Sufficient (1)
The above-mentioned definition of outstanding was only an example in order to assist the panellists in the positioning of the classification. There was no exact definition for the category fitness.
5 The panels discussed the category fitness and made the final conclusions of the interpretation of it.
13
1.8 Timetable of the evaluation
The main timetable of the evaluation:
1. Registration November 2010
2. Submission of self-evaluation materials January–February 2011
3. External peer review May–September 2011
4. Published reports March–April 2012
- University level public report - RC specific reports
The entire evaluation was implemented during the university’s strategy period 2010–2012. The preliminary results were available for the planning of the following strategy period in late autumn 2011. The evaluation reports will be published in March/April 2012. More detailed time schedule is published in the University report.
1.9 Evaluation feedback – consensus of the entire panel
The panellists evaluated all the RC-specific material before the meetings in Helsinki and mailed the draft reports to the evaluation office. The latest interim versions were on-line available to all the panellists on the Wiki-sites. In September 2011, in Helsinki the panels discussed the material, revised the first draft reports and decided the final numeric evaluation. After the meetings in Helsinki, the panels continued working and finalised the reports before the end of November 2011. The final RC-specific reports are the consensus of the entire panel.
The evaluation reports were written by the panels independently. During the editing process, the evaluation office requested some clarifications from the panels when necessary. The tone and style in the reports were not harmonized in the editing process. All the reports follow the original texts written by the panels as far as it was possible.
The original evaluation material of the RCs, provided for the panellists is attached at the end of the report. It is essential to notice that the exported lists of publications and other scientific activities depend how the data was stored in the TUHAT-RIS by the RCs.
14
15
2 Evaluation feedback
2.1 Focus and quality of the RC’s research
Description of
the RC’s research focus
the quality of the RC’s research (incl. key research questions and results)
the scientific significance of the RC’s research in the research field(s)
Identification of the ways to strengthen the focus and improve the quality of the RC’s research ASPECTS: Scientific quality, scientific significance, societal impact, innovativeness
CellMolBiol is composed of senior and junior groups all located at the Institute of Biotechnology. The subject areas that the RC is engaged in span important areas in contemporary cell biology. Also virology is included and this is an ideal extension since viruses have proven to be important tools in molecular cell biological research. A strength of the RC is that the junior group leaders are well chosen with a lot of promise and potential. Cell biology is an integrative area of modern biology and therefore plays an important role in shaping modern biological research. The RC represents the cutting edge of its field and thus strengthens the international attractiveness of the Viikki Campus. The publication record is impressive and demonstrates the strengths of this RC.
Numeric evaluation: 5 (Outstanding)
2.2 Practises and quality of doctoral training
Organising of the doctoral training in the RC. Description of the RC’s principles for:
recruitment and selection of doctoral candidates
supervision of doctoral candidates
collaboration with faculties, departments/institutes, and potential graduate schools/doctoral programmes
good practises and quality assurance in doctoral training
assuring of good career perspectives for the doctoral candidates/fresh doctorates
Identification of the RC’s strengths and challenges related to the practises and quality of doctoral training, and the actions planned for their development.
Additional material: TUHAT compilation of the RC’s other scientific activities/supervision of doctoral dissertations
ASPECTS: Processes and good practices related to leadership and management
During the years 2005-2010, at total of 17 students defended their PhD thesis and there are currently 19 students in the RC. All of the doctoral students completing their theses in the past period have obtained positions utilizing their training. The recruitment is centralized and thus up to present top standards. More than 50% of the present PhD students are foreigners, attesting to the attractiveness of the RC. The RC runs several important courses and thus constitutes an asset to the overall quality of the Viikki Campus. The training could further strengthened by requiring more frequent meetings of the PhD "committees", perhaps twice a year.
Numeric evaluation: 5 (Outstanding)
16
2.3 The societal impact of research and doctoral training
Description on how the RC interacts with and contributes to the society (collaboration with public, private and/or 3rd sector).
Identification of the ways to strengthen the societal impact of the RC’s research and doctoral training.
Additional material: TUHAT compilation of the RC’s other scientific activities.
ASPECTS: Societal impact, national and international collaboration, innovativeness
The Pls in the RC are actively working in outreach activities to spread their research messages to the lay public. The societal impact could be increased by instituting “Science and Society” discussions in their programme. The PhD students need to actively consider the impact of their research on society.
Numeric evaluation: 4 (Excellent)
2.4 International and national (incl. intersectoral) research collaboration and researcher mobility
Description of
the RC’s research collaborations and joint doctoral training activities
how the RC has promoted researcher mobility
Identification of the RC’s strengths and challenges related to research collaboration and researcher mobility, and the actions planned for their development.
ASPECTS: Scientific quality, national and international collaboration
The RC has a large network of international contacts and collaborations. Already the fact they harbor so many foreign PhD students increases their international impact. Also collaborations within the RC are increasing, making the RC even more attractive for applicants. The two new recruitments also strengthen the RC to the inside and to the outside.
Numeric evaluation: 5 (Outstanding)
2.5 Operational conditions
Description of the operational conditions in the RC’s research environment (e.g. research infrastructure, balance between research and teaching duties).
Identification of the RC’s strengths and challenges related to operational conditions, and the actions planned for their development.
ASPECTS: Processes and good practices related to leadership and management
The RC is well supported and has a state-of-the-art infrastructure at its disposal. The Light Microscopy Unit and the Electron Microscopy Unit are real strengths and need to be kept up to date. The RC plans to increase its use of high-throughput analysis technologies and screening capacities. These are very expensive areas with variable output. Great care should be taken not move into areas already overcrowded internationally and difficult to finance.
2.6 Leadership and management in the researcher community
Description of
the execution and processes of leadership in the RC
how the management-related responsibilities and roles are distributed in the RC
how the leadership- and management-related processes support
17
high quality research
collaboration between principal investigators and other researchers in the RC
the RC’s research focus
strengthening of the RC’s know-how
Identification of the RC’s strengths and challenges related to leadership and management, and the actions planned for developing the processes
ASPECTS: Processes and good practices related to leadership and management
The leader of the RC Pekka Lappalainen is doing an excellent job to promote this RC. Important is also that the overall quality of the PIs is unusual high. One reason for this is the stringent recruitment process that the Institute of Biotechnology has in place to ensure quality of its incoming PIs.
Furthermore, the regular 4-year reviews by Scientific Advisory Board and external reviewers of PIs and their research is excellent and should be used more widely across the campus to sharpen the research profile.
2.7 External competitive funding of the RC
• The RCs were asked to provide information of such external competitive funding, where:
• the funding decisions have been made during 1.1.2005–31.12.2010, and
• the administrator of the funding is/has been the University of Helsinki
• On the e-form the RCs were asked to provide:
1) The relevant funding source(s) from a given list (Academy of Finland/Research Council, TEKES/The Finnish Funding Agency for Technology and Innovation, EU, ERC, foundations, other national funding organisations, other international funding organizations), and
2) The total sum of funding which the organisation in question had decided to allocate to the RCs members during 1.1.2005–31.12.2010.
Competitive funding reported in the text is also to be considered when evaluating this point.
ASPECTS: Scientific quality, scientific significance, societal impact, innovativeness and future significance The overall funding is quite low, only around 7.5 MEUR for a large grouping. Here some work will be necessary to increase the grant flow, for instance EU funding could be augmented by the excellent infrastructure that the RC has at its disposal.
2.8 The RC’s strategic action plan for 2011–2013
• RC’s description of their future perspectives in relation to research and doctoral training.
ASPECTS: Scientific quality, scientific significance, societal Impact, processes and good practices related to leadership and management, national and international collaboration, innovativeness, future significance The RC is doing its homework in an outstanding way and their strategic plan does not need much comment. They are well prepared for the future and CellMolBiol is one of the flagship RCs.
2.9 Evaluation of the category of the RC in the context of entity of the evaluation material (1-8)
The RC’s fitness to the chosen participation category.
Category 1. The research of the participating community represents the international cutting edge in its field.
18
The RC has chosen as its participation category: 1 ‘Research of participating community represents the international cutting edge of its field’. This quality of the RC corresponds well to this category and they are also well prepared to maintain the outstanding status that they have reached.
Numeric evaluation: 5 (Outstanding)
2.10 Short description of how the RC members contributed the compilation of the stage 2 material
All members contributed.
2.11 How the UH’s focus areas are presented in the RC’s research
Focus area 2: The basic structure of life
The research with the RC is in line with the focus areas of the University of Helsinki.
2.12 RC-specific main recommendations
—
2.13 RC-specific conclusions
—
19
3 Appendices
A. Original evaluation material
a. Registration material – Stage 1
b. Answers to evaluation questions – Stage 2 c. List of publications
d. List of other scientific activities B. Bibliometric analyses
a. Analysis provided by CWTS/University of Leiden b. Analysis provided by Helsinki University Library (66 RCs)
International evaluation of research and doctoral training at the University of Helsinki 2005-2010
RC-SPECIFIC MATERIAL FOR THE PEER REVIEW
NAME OF THE RESEARCHER COMMUNITY:
The Research Program in Cell and Molecular Biology (CellMolBiol) LEADER OF THE RESEARCHER COMMUNITY:
Professor Pekka Lappalainen, Institute of Biotechnology
RC-SPECIFIC MATERIAL FOR THE PEER REVIEW:
Material submitted by the RC at stages 1 and 2 of the evaluation
- STAGE 1 material: RC’s registration form (incl. list of RC participants in an excel table) - STAGE 2 material: RC’s answers to evaluation questions
TUHAT compilations of the RC members’ publications 1.1.2005-31.12.2010
TUHAT compilations of the RC members’ other scientific activities 1.1.2005-31.12.2010
Web of Science(WoS)-based bibliometrics of the RC’s publications data 1.1.2005-31.12.2010 (analysis carried out by CWTS, Leiden University)
NB! Since Web of Science(WoS)-based bibliometrics does not provide representative results for most RCs representing humanities, social sciences and computer sciences, the publications of these RCs will be analyzed by the UH Library (results available by the end of June, 2011)
1 INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL TRAINING AT THE
UNIVERSITY OF HELSINKI
RC-SPECIFIC STAGE 1 MATERIAL (registration form)
Name: Lappalainen, Pekka E-mail:
Phone: 09-19159499
Affiliation: Institute of Biotechnology Street address: Viikinkaari 9
Name of the participating RC (max. 30 characters): The Research Program in Cell and Molecular Biology Acronym for the participating RC (max. 10 characters): CellMolBiol
Description of the operational basis in 2005-2010 (eg. research collaboration, joint doctoral training activities) on which the RC was formed (MAX. 2200 characters with spaces): The Research Program in Cell and Molecular Biology (CellMolBiol) is one of the four research programs of the Institute of Biotechnology (see: http://www.biocenter.helsinki.fi/bi/research/index.html). The research within this program focuses on analyzing basic principles of cell and organelle morphogenesis and dynamics, cell growth and metabolism, interactions of viruses with host cells as well as on elucidating the mechanisms by which specific processes form and function in highly specialized mammalian cells such as neurons. The research groups use mammalian and yeast cells as well as the C. elegans nematode and Drosophila fruit fly as model systems in these studies. The program also harbors two modern biological imaging centers: Light
Microscopy Unit for advanced light microscopy studies on live cells and Electron Micrsocopy Unit, which both are national technology platforms for advanced biological imaging through Biocenter Finland organization.
The Program in Cell and Molecular Biology currently consist of seven research groups whose research interests range from cytoskeletal and membrane dynamics to the regulation of transcription and neuronal activity. In addition, two PIs who where members of the research program (Oscar Puig and Harri Savilahti) have been appointed to positions outside the institute during the years 2005-2010. The researcher community organizes joint seminar series, journal club, lecture/practical courses, and regular group leaders’ meetings. In addition, several groups within this community are collaborating with each other, as indicated by a large number of joint publications and currently ongoing collaborations. Thus, CellMolBiol program is a truly cohesive researcher community.
Main scientific field of the RC’s research: biological, agricultural and veterinary sciences RC's scientific subfield 1: Cell Biology
1RESPONSIBLE PERSON
2DESCRIPTION OF THE PARTICIPATING RESEARCHER COMMUNITY (RC)
3SCIENTIFIC FIELDS OF THE RC
2 INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL TRAINING AT THE
UNIVERSITY OF HELSINKI
RC-SPECIFIC STAGE 1 MATERIAL (registration form) RC's scientific subfield 2: Biochemistry and Molecular Biology
RC's scientific subfield 3: --Select-- RC's scientific subfield 4: --Select-- Other, if not in the list:
Participation category: 1. Research of the participating community represents the international cutting edge in its field
Justification for the selected participation category (MAX. 2200 characters with spaces): The Research Program in Cell and Molecular Biology is a mixture of senior and junior groups, which all are internationally well recognized and carry out top-quality research in their fields. For example, the research director of the program is among the world leading experts on actin and membrane dynamics research and is currently an executive editor of the journal Cytoskeleton (published by Wiley-Blackwell). The laboratories of Claudio Rivera, Eija Jokitalo, Jussi Jäntti and Tero Ahola are internationally recognized experts on molecular neurobiology, organelle biology, secretion and Semliki Forest virus biology, respectively. The two junior PIs the program (Vartiainen and Hietakangas) were recruited to the program after a successful international postdoctoral training. The high quality of the research is also indicated by the fact that the groups of this program regularly publish articles in top international journals such as Science, Neuron, Developmental Cell, Genes and Development, Current Biology, Journal of Cell Biology, and PNAS. The funding of the research within the RC is mainly from the highly competitive external sources. Furthermore, the core facilities administrated by this researcher community harbor several cutting edge technologies such as electron tomography, correlative light-electron microscopy, light microscopy analysis of protein dynamics in live cells, and interaction studies using fluorescence life-time imaging.
Public description of the RC's research and doctoral training (MAX. 2200 characters with spaces): The Research Program in Cell and Molecular Biology is one of the four research programs of the Institute of Biotechnology. The researchers of this program are interested in basic principles of cell and organelle morphogenesis and dynamics. These include for example analysis of the mechanisms of cell migration, morphogenesis, vesicle transport, and cell growth and metabolism as well as gene expression. In addition, the research groups within this program are elucidating the mechanisms by which specific processes form and function in highly specialized mammalian cells such as neurons, and how viruses interact with their host cells during the infection processes. It is important to note that defects in many of these processes are linked to diseases such as cancer, diabetes and neurodegenerative disorders. Thus, in addition to
fundamental information concerning the mechanisms of central cellular processes, these studies also aim to open new avenues for diagnosis or treatment of these diseases.
The research groups of this program are also very active in doctoral training. Currently 22 students are carrying out PhD-thesis projects in the groups. Furthermore, this researcher community organizes a journal club, seminar series and several practical and lecture courses for doctoral students and post-docs. These
4RC'S PARTICIPATION CATEGORY
5DESCRIPTION OF THE RC'S RESEARCH AND DOCTORAL TRAINING
3 INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL TRAINING AT THE
UNIVERSITY OF HELSINKI
RC-SPECIFIC STAGE 1 MATERIAL (registration form)
courses/events are open for all biosciences/medicine doctoral students and will thus not only benefit the students working in the research groups of this researcher community but also other students across the campus boundaries at University of Helsinki.
Significance of the RC's research and doctoral training for the University of Helsinki (MAX. 2200 characters with spaces): The director if the Research Program in Cell and Molecular Biology (Pekka Lappalainen) is the head of the one of the largest graduate programs at University of Helsinki. The Helsinki Graduate Program in Biotechnology and Molecular Biology (GPBM) is an international Ph.D. training program administered by the Biocentrum Helsinki organization and the Institute of Biotechnology, University of Helsinki. The program has currently 50 students supported by the Ministry of Culture and Education. It organizes a large number of practical and theoretical courses for doctoral students, and supports their thesis projects/studies through travel funds and formal thesis committee work. In addition, a member of the RC (Eija Jokitalo) serves in the board of the Viikki Doctoral Program in Molecular Biosciences (VGSB), representing the field of cell biology. All research groups within the Research Program in Cell and Molecular Biology are actively organizing courses to the GPBM and VGSB graduate programs as well as other graduate programs operating at University of Helsinki. Thus, in addition to training doctoral students in their own laboratories, the senior scientists of this researcher community actively participate in the training of other doctoral students at our university through organizing courses to graduate programs and participating in the administration of graduate programs.
Keywords: Cell biology, molecular biology, cytoskeleton, transcription, cell growth, endoplasmic reticulum, virus, neuron, secretion, microscopy
Justified estimate of the quality of the RC's research and doctoral training at national and international level during 2005-2010 (MAX. 2200 characters with spaces): This researcher community carries out top international quality research in the fields of cell and molecular biology. The high quality of the research is indicated e.g. by the fact that the groups of this program regularly publish articles in top international journals such as Science, Neuron, Developmental Cell, Genes and Development, Current Biology, Journal of Cell Biology, and PNAS. Each group within the RC has constantly international collaborations. The RC has been successful in obtaining research funding from competitive sources such as Academy of Finland, Sigrid Juselius Foundation, Finnish Cancer Foundation, Biocentrum Helsinki etc. Furthermore, the core facilities administrated by this researcher community harbor several cutting edge technologies such as electron tomography, correlative light-electron microscopy, light microscopy analysis of protein dynamics in live cells, and interaction studies using fluorescence life-time imaging. Imaging core facilities provide services and collaboration nationwide and have active ongoing international collaborations.
Comments on how the RC's scientific productivity and doctoral training should be evaluated (MAX. 2200 characters with spaces): In the biosciences field, top quality science is typically published in the best international journals. Furthermore, the most important studies are typically highly cited. Thus, the best methods for assessing the quality of our research are to analyze the quality of the journals where the research was published and examine the number of citations to these articles.
6QUALITY OF RC'S RESEARCH AND DOCTORAL TRAINING
4 INTERNATIONAL EVALUATION OF RESEARCH AND DOCTORAL TRAINING AT THE
UNIVERSITY OF HELSINKI
RC-SPECIFIC STAGE 1 MATERIAL (registration form)
In assessing the quality of doctoral training, one should not only pay attention to the number of doctoral theses produced by the researcher community, but also pay attention to the quality of doctoral theses (using the same parameters as described above) and examine the time (years) spend for carrying out the thesis projects.
