AUDIT OF THE UNIVERSITY OF OULU 2018

(1)

AUDIT OF THE UNIVERSITY OF OULU 2018

Publications 3:2018Audit of the University of Oulu 2018

Ellen Hazelkorn Rediet Abebe

Heidi Ahokallio-Leppälä Philippe Bouillard

Kati Korhonen-Yrjänheikki Touko Apajalahti

Anni Tuurnas

(2)

AUDIT OF THE UNIVERSITY OF OULU 2018

Ellen Hazelkorn Rediet Abebe Heidi Ahokallio-Leppälä Philippe Bouillard Kati Korhonen-Yrjänheikki Touko Apajalahti Anni Tuurnas

Finnish Education Evaluation Centre Publications 3:2018

(3)

PUBLISHER Finnish Education Evaluation Centre

BOOK DESIGN Juha Juvonen (org.) & Sirpa Ropponen (edit) LAYOUT Juvenes Print – Suomen Yliopistopaino Oy, Tampere ISBN 978-952-206-429-5 (pb)

ISBN 978-952-206-430-1 (pdf) ISSN 2342-4176 (paperback) ISSN 2342-4184 (pdf) ISSN-L 2342-4176

PRINTED BY Juvenes Print – Suomen Yliopistopaino Oy, Tampere 2018

(4)

Abstract

Published by

Finnish Education Evaluation Centre (FINEEC) Name of Publication

Audit of University of Oulu 2018 Authors

Ellen Hazelkorn, Rediet Abebe, Heidi Ahokallio-Leppälä, Philippe Bouillard, Kati Korhonen-Yrjänheikki, Touko Apajalahti and Anni Tuurnas

The Finnish Education Evaluation Centre has conducted an audit of the University of Oulu and has awarded the institution a quality label that is valid for six years from 28 February 2018. The quality system of the University of Oulu fulfils the national criteria set for the quality management of higher education institutions, and corresponds to the European quality assurance principles and recommendations for higher education institutions.

The object of the audit was the quality system that the University has developed based on its own needs and goals. The freely selected audit target chosen by the University was the “management of personnel well-being in a change”.

The following were regarded as key strengths of the quality system:

▪ The university has systematically developed a quality system, which is strategically led by the Executive Group, with good participation from faculty, support staff, students and external stakeholders.

▪ There is a good working relationship and atmosphere in the university, with high levels of co-operation and trust, and professional support for staff and students throughout the change processes.

▪ The university’s research strategy with its five focus areas, along with the reorganisation of its structure, has been based upon a comprehensive assessment of research expertise, drawing on the competence base survey for research groups and linked with global and regional challenges.

(5)

Among others, the following recommendations were made to the University:

▪ The university should pay attention to making the quality system more consistent across the organisational units and more comparable between the core duties.

▪ The university should consider developing a more effective, inclusive and integrated internal communications system, appropriate for students and staff, to enhance coherent information sharing and dialogue.

▪ Greater attention should be given to improving pedagogic practices to ensure an enhanced student-centred learning experience for all students and to tackling challenges with student completion.

Keywords

Audit, evaluation, higher education institutions, quality, quality management, quality system, university

(6)

Tiivistelmä

Julkaisija

Kansallinen koulutuksen arviointikeskus (Karvi) Julkaisun nimi

Audit of University of Oulu 2018 (Oulun yliopiston auditointi 2018) Tekijät

Ellen Hazelkorn, Rediet Abebe, Heidi Ahokallio-Leppälä, Philippe Bouillard, Kati Korhonen-Yrjänheikki, Touko Apajalahti ja Anni Tuurnas

Kansallinen koulutuksen arviointikeskus on toteuttanut Oulun yliopiston auditoinnin ja antanut korkeakoululle laatuleiman, joka on voimassa kuusi vuotta 28.2.2018 alkaen. Oulun yliopiston laatujärjestelmä täyttää korkeakoulujen laadunhallinnalle asetetut kansalliset kriteerit ja vastaa eurooppalaisia korkeakoulujen laadunhallinnan periaatteita ja suosituksia.

Auditoinnin kohteena oli Oulun yliopiston laatujärjestelmä, jonka yliopisto on kehittänyt omista lähtökohdistaan ja tavoitteidensa mukaisesti. Yliopiston valitsema vapaavalintainen auditointi- kohde oli henkilöstön hyvinvoinnin johtaminen muutoksessa.

Auditoinnin perusteella laatujärjestelmän keskeisiä vahvuuksia ovat:

▪ Yliopisto on kehittänyt laatujärjestelmää systemaattisesti. Johtoryhmä johtaa laatujärjes- telmää strategisesti ja henkilöstö, opiskelijat ja ulkoiset sidosryhmät osallistuvat hyvin sen toteuttamiseen.

▪ Hyvä työskentelyilmapiiri, johon kuuluvat korkea yhteistyön ja luottamuksen määrä sekä ammattimainen henkilöstön ja opiskelijoiden tukeminen läpi muutosprosessien.

▪ Viiteen painopisteeseen keskittyvän yliopiston tutkimusstrategian laatiminen ja tutki- musorganisaation uudistus ovat perustuneet kattavaan tutkimuksen arviointiin, tutki- musryhmien osaamisen selvittämiseen ja sekä globaalien että alueellisten haasteiden huomiointiin.

(7)

Yliopistolle annettiin muun muassa seuraavia kehittämissuosituksia:

▪ Yliopiston tulisi kiinnittää enemmän huomiota laatujärjestelmän kehittämiseksi yhdenmu- kaisemmaksi organisaation eri yksiköissä ja vertailtavammaksi yliopiston perustehtävien välillä.

▪ Johdonmukaisen viestinnän ja vuorovaikutuksen edistämiseksi yliopiston tulisi harkita vaikuttavamman, osallistavamman ja integroidumman sisäisen viestinnän järjestelmän kehittämistä, joka soveltuisi sekä opiskelijoille että henkilöstölle.

▪ Pedagogisten käytäntöjen kehittämiseen tulisi kiinnittää suurempaa huomiota opiske- lijakeskeisemmän opiskelukokemuksen takaamiseksi kaikille opiskelijoille ja opintojen läpäisyyn liittyvien haasteiden ratkaisemiseksi.

Avainsanat

Auditointi, arviointi, korkeakoulut, laatu, laadunhallinta, laatujärjestelmä, yliopisto

(8)

Sammandrag

Utgivare

Nationella centret för utbildningsutvärdering (NCU) Publikation

Audit of University of Oulu 2018 (Auditering av Uleåborgs universitet 2018) Författare

Ellen Hazelkorn, Rediet Abebe, Heidi Ahokallio-Leppälä, Philippe Bouillard, Kati Korhonen-Yrjänheikki, Touko Apajalahti och Anni Tuurnas

Nationella centret för utbildningsutvärdering har genomfört en auditering av Uleåborgs universitet och har beviljat universitetet en kvalitetsstämpel som gäller i sex år från och med den 28 februari 2018. Universitetets kvalitetssystem uppfyller de nationella kriterierna för kvalitetshantering som fastställts för högskolor och motsvarar de europeiska principerna för och rekommendationerna om högskolornas kvalitetshantering.

Föremål för auditeringen var Uleåborgs universitets kvalitetssystem som universitetet tagit fram utifrån sina egna utgångspunkter och mål. Auditeringsobjektet som universitetet kunde fritt välja var var ledning av personalens välbefinnande vid förändringar.

Kvalitetssystemets viktigaste styrkor är:

▪ Universitetet har utvecklat sitt kvalitetssystem systematiskt. Ledningsgruppen leder kvalitetssystemet strategiskt. Personal, studerande och externa intressenter deltar aktivt i implementeringen av systemet.

▪ Gott arbetsklimat, som inkluderar omfattande samarbete och stort förtroende samt stöd till personal och studerande genom förändringsprocesserna som är professionellt ordnat.

▪ Utarbetandet av universitetets forskningsstrategi som betonar fem fokusområden och förnyelsen av forskningsorganisationen har grundat sig på omfattande utvärdering av forskningen, utredning av forskningsgruppernas kompetens och beaktande av såväl globala som regionala utmaningar.

(9)

Bland annat följande rekommendationer framläggs för Uleåborgs universitet:

▪ Universitetet bör fästa mer uppmärksamhet vid att utveckla kvalitetssystemet så att det fungerar jämnare i alla enheter och att systemet blir mer jämförbart mellan de olika grundläggande uppgifterna.

▪ För att främja en konsekvent kommunikation och växelverkan bör universitetet överväga att utveckla ett mer effektivt, inkluderande och integrerat internt kommunikationssystem som lämpar sig för både studerande och personal.

▪ Vid utveckling av de pedagogiska metoderna bör större uppmärksamhet fästas vid att garantera en mer studentcentrerad lärupplevelse för alla studerande och vid att lösa utma- ningarna med genomströmning.

Nyckelord

Auditering, högskolor, kvalitet, kvalitetshantering, kvalitetssystem, universitet, utvärdering

(10)

Audit targets 1

and process

1.1 Audit targets

The target of the audit is the quality system that the University of Oulu has developed based on its own needs and goals. The focus of the audit were the procedures and processes that the institution uses to maintain, develop and enhance the quality of its operations. In accordance with the principle of enhancement-led-evaluation, the audit did not evaluate the university’s objectives, the content of its activities or its results. The aim of the audit is to help the institution to identify strengths, good practice and areas in need of development in its operations.

Audits by the Finnish Education Evaluation Centre (FINEEC) evaluate whether an institution’s quality system meets the national criteria (Appendix 1) and whether it corresponds to the Standards and Guidelines for Quality Assurance in the European Higher Education Area¹ (ESG).

Furthermore, the audit evaluates how well the quality system meets strategic and operations management needs, as well as the quality management of the university’s core duties and the extent to which it is comprehensive and effective. In addition, FINEEC audits focus on evaluating the institution’s quality policy, the development of the quality system, as well as how effective and dynamic an entity the system forms.

The University of Oulu chose the “management of personnel well-being in a change” as its optional audit target. As samples of first- and second cycle degree education, the university chose the Bachelor’s Degree Programme in Teaching and Education and the Master’s Degree Programmes in Primary Teacher Education and in Early Childhood Education. As a sample of doctoral education, the university chose the Doctoral degree programme of Biochemistry and Molecular Medicine.

The audit team chose the Bachelor’s and Master’s degree programme in Information Processing Science as the third sample of degree education.

1 Standards and Guidelines for Quality Assurance in the European Higher Education Area are available at http://www.enqa.

eu/index.php/home/esg/

(13)

The audit targets for the University of Oulu were:

1. The quality policy of the higher education institution 2. The quality system’s link with strategic management 3. Development of the quality system

4. Quality management of the higher education institution’s core duties a. Degree education (including first-, second- and third-cycle education)² b. Research, development and innovation activities (RDI)

c. The societal impact and regional development work³

d. Optional audit target: Management of personnel well-being in a change 5. Samples of degree education:

i. Bachelor’s degree in Teaching and Education and master’s degrees in Primary Teacher Education and in Early Childhood Education

ii. Doctoral degree programme of Biochemistry and Molecular Medicine

iii. Bachelor’s and master’s degree programme in Information Processing Science 6. The quality system as a whole.

A set of criteria that is based on a scale of four development stages of quality management (absent, emerging, developing and advanced) was employed in the audit. The development stages have been specified for each audit target and they are determined individually for each audit target. The optional audit target is not taken into account when evaluating whether the audit will pass.

1.2 Implementation of the audit

The audit is based on the basic material and self-evaluation report submitted by the University of Oulu as well as an audit visit to the University on 17–19 October 2017. The audit team also had access to electronic materials that were important for quality management. The main phases and timeframe of the audit process are listed in Appendix 2.

An international audit team carried out the audit in English. The University was given the opportunity to comment on the team’s composition, particularly from the perspective of disqualification prior to the appointment of the audit team.

2 First-cycle degrees include bachelor’s degrees and second-cycle degrees include master’s degrees. Third-cycle degrees include postgraduate licentiate and doctoral degrees.

3 Including social responsibility, continuing education, open university education, as well as paid-services education

(14)

The audit team:

Ellen Hazelkorn, former Policy Advisor to the Higher Education Authority, and Emeritus Professor and Director, Higher Education Policy Research Unit, Dublin Institute of Technology, Ireland (Chair of the audit team)

Kati Korhonen-Yrjänheikki, Managing Director, the Education Fund, Finland (Vice-chair of the audit team)

Rediet Abebe, Doctoral student, Higher Education Group, University of Tampere, Finland Heidi Ahokallio-Leppälä, Vice Rector, Häme University of Applied Sciences, Finland

Philippe Bouillard, Head of Civil, Architectural and Urban Engineering Dept., Université Libre de Bruxelles, Belgium

Senior Advisor Touko Apajalahti from FINEEC acted as the project manager for the audit, and Planning Officer Anni Tuurnas from FINEEC acted as the backup for the project manager.

As indicated, the audit team conducted a three-day audit visit to the University. The purpose of the visit was to verify and supplement the observations made of the quality system based on the audit material. The programme of the visit is shown in Appendix 3.

The audit team drew up this report based on the material accumulated during the evaluation and on the analysis of that material. The audit team members produced the report jointly by drawing on the expertise of each team member. The University of Oulu was given the opportunity to check the report for factual information prior to the FINEEC Higher Education Evaluation Committee’s decision-making meeting.

(15)

2 The organisation of the University of Oulu

The University of Oulu was founded in 1958. The university defines itself as an international science university which creates innovation for the future, as well as well-being and knowledge through multidisciplinary research and education.

The university pursues a mission that aims at promoting free research, education and culture of an internationally high standard, building competence that increases well-being, and safeguarding the availability of a highly educated labour force and researchers in its area of influence. In this strategy, emphasis is placed at promoting high quality, multidisciplinary and cooperative research and innovation that enable making significant contribution to solving global challenges in five key focus areas. These focus areas address creating sustainability through materials and systems, molecular and environmental basis for lifelong health, digital solutions in sensing and interactions, earth and near-space system and environmental change, and understanding humans in change situations.

The university has two campuses that are both located in the City of Oulu.

The University of Oulu has ten faculties:

▪ the Faculty of Biochemistry and Molecular Medicine ▪ the Faculty of Education

▪ the Faculty of Humanities

▪ the Faculty of Information Technology and Electrical Engineering ▪ the Faculty of Science

▪ the Faculty of Medicine ▪ the Faculty of Technology ▪ Oulu Business School ▪ Oulu Mining School, and ▪ Oulu School of Architecture

(16)

In addition, the university coordinates the operations of two off-campus institutes: The Kerttu Saalasti Institute in the municipality of Nivala, and the Kajaani University Consortium which is a consortium of four networked universities (University of Oulu, University of Eastern Finland, University of Jyväskylä and University of Lapland) that have units in the cities of Kajaani and Sotkamo.

The organisation within the faculties is based on research units. Each faculty is led by a Dean, and each research unit is led by a Head of a research unit. Degree education is organised into degree programmes. A faculty education committee oversees the bachelor’s and master’s degree programmes in each faculty. Regarding doctoral education, in addition to the individual doctoral programmes, the university has a Graduate School that coordinates doctoral education at the university level.

The organisation of the University is described in the Figure 1:

MEDICINE

RECTOR Jouko Niinimäki

BOARD UNIVERSITY COLLEGIUM

BIOCHEMISTRY & MOLECULAR BIOLOGY NATURAL SCIENCE ARCHITECTURE HUMANITIES BUSINESS SCHOOL EDUCATION TECHNOLOGY

INFORMATION TECHNOLOGY & ELECTRICAL ENGINEERING OULU MININGSCHOOL VICE RECTOR

RESEARCH VICE RECTOR

EDUCATION HALLINTOJOHTAJA

Essi Kiuru

CHIEF FINANCIAL

OFFICER COMMUNICATIONS DIRECTOR VICE RECTOR

CO-OPERATION

ADMINISTRATIVE DIRECTOR

Kajaani University Consortium Kerttu Saalasti Institute Sodankylä Geophysical

Observatory

HUMAN RESOURCES DIRECTOR

Kokkola University Consortium Chydenius FACULTIES

FIGURE 1. Organisation of the University of Oulu.

(17)

Table 1 presents key facts and figures of University of Oulu

TABLE 1. Key facts and figures of University of Oulu.

Students (full-time equivalent, 2016) Number

Bachelor’s degree 5449

Master’s degree 3689

Licentiate degree 9

Doctoral degree 618

Degrees awarded (annual average 2014–2016)

Bachelor’s degree 1322

Master’s degree 1361

Licentiate degree 5

Doctoral degree 180

Staff (full-time equivalent, 2016)

Professors 200

Other teaching and research personnel 1440

Other staff 896

(18)

3

The quality policy

The quality policy’s rationale, objectives and the overall division of responsibilities are clearly defined as a result of an inclusive process. The quality policy is accessible to internal and external stakeholders and the university has channels to support the communication of the quality policy. The information needs of different stakeholder groups should still be addressed more carefully. The quality policy is well-aligned with the university’s strategy.

The quality policy of the University of Oulu is at a developing stage.

3.1 Rationale, objectives and division of responsibility

The quality system of the University of Oulu is built on a comprehensive and supportive quality policy. The policy sets guaranteeing effectiveness, productivity and fluency in education, research and interaction with society in keeping with the university’s strategy as the overarching objective of the quality system.

The quality policy promotes a participatory approach towards quality management. There is no separate unit responsible for quality management, instead quality management is perceived and practised as a shared responsibility. The University of Oulu operates with an integrated quality system where the responsibilities for quality management are integrated with everyday work and tied to the division of responsibilities defined for managerial positions. The quality policy of the University of Oulu establishes a strong link between quality management and operations management.

The quality system is based on the principle of continuous improvement of operations according to a Plan-Do-Check-Act (PDCA) cycle which is presented in Figure 2. This cycle, which is at the core of the quality system, provides the framework and procedures for planning, implementing, monitoring and improving operations at all levels of the university. The framework is suitable for application at university, unit and individual levels.

(19)

FIGURE 2. The quality system of the University of Oulu.

The quality policy defines key principles that guide the quality management system. These principles focus on the notion of quality management as an integral component of the university’s operating culture; encompassing all functions and actors of the university community; improving operations through feedback, evaluations and indicator data; producing information useful in assessing effectiveness, productivity and fluency of operations; setting objectives, implementing actions and assessing results; supporting improvements in education, research and other activities;

supporting the spreading of good practices; and joint designing and utilising of quality management procedures, processes and systems. The policy provides guidelines for quality management in areas of strategic management, education, research, interaction with society, support services and resource management. The comprehensive scope of the policy enables enhancing quality work in all areas of operation and levels of the university structure.

(20)

Based on the audit interviews, the university community and its external stakeholders are aware of the quality policy and broadly understand the main purpose of the quality policy, especially the principles of continuous improvement of operations and shared responsibilities for quality management. However, the University could still clarify and improve the definition of key concepts and principles embedded in the quality policy, including the concept of quality itself, as well as how this is defined and what it means for the University, in a manner that is easily understandable for actors and relatable to their daily activities. This is important as a clear definition of quality could not be found either in the recently approved quality manual or in the descriptions of the quality system, even though the university does have indicators to assess the quality of education, research and interaction with society. Although the academic staff seem to agree that the quality policy and quality manual have become clearer after the previous audit, they also indicated in the audit interviews that the quality policy is somewhat abstract and remote to the daily work.

Responsibilities for quality management are defined mainly in the quality policy. The Rector is in charge of overseeing the quality management at the university. The policy assigns quality management responsibilities mainly to the Vice Rectors of education, research and co-operation, Deans, Education Deans, Heads of research units, Programme Directors and Heads of units.

Staff holding managerial positions are tasked with the responsibility of overseeing the quality management of specific operations undertaken in their units. In addition to specifying what is expected of such key actors, the division of responsibilities for quality management exists at units.

The quality policy is complemented by the other documents and tools that the university uses to define quality management responsibilities. These include the university regulations, financial reporting tools, annual calendars, the management system document, and other frameworks that guide staff on how to conduct operations with high quality. The university’s operations are structured as a process and the core processes are research, education and interaction with society.

Process descriptions describe the university’s operations and the Information Management System contains a process map that illustrates the university’s actions as a whole. Based on the audit visit, the operational instructions are the key aspects for defining the responsibilities for quality work.

However, the university could still improve the clarity and concreteness of division of responsibilities for quality management by providing more detailed descriptions of the roles and responsibilities that individual academic staff, students and external stakeholders have in relation to quality management.

The audit interviews conducted with academic staff indicated that the responsibilities they have at the individual level for quality management do not seem to be always clearly communicated to them. It is also important to clearly specify in what ways external stakeholders are expected to participate in quality management.

The audit interviews confirmed that internal and external stakeholders participated actively in the processes of formulating, evaluating and developing the quality policy through extensive discussion sessions where, for instance, academic staff, students, management, the Board of Directors, and external stakeholders provided inputs for the draft quality policy.

(21)

3.2 Communication of the quality policy

The university’s intranet Notio is the primary platform where information regarding the quality system and its basic principles and procedures are communicated. Quality system documentation includes the university’s quality policy, action plans, operating policies and operating instructions accessible on the intranet and the quality manual. The quality manual adds further detail to the descriptions of the quality system and quality management practices defined in the quality policy.

The aim of the quality manual is to translate the quality system to concrete terms and link the principles and practices of quality management. Some of the documentation can also be found in the university’s operative information systems.

Based on the audit material, the university has put in place mechanisms that are capable of facilitating active communication of the quality policy, objectives and principles of quality management, and strategically important activities. Information is communicated through, for instance, internal communication channels, information systems, management forums, university regulations, management training courses and other platforms. In addition, email and organised coffee-room conversations are other channels of communication frequently used by staff. Despite the existence of these mechanisms, based on the interviews, there are still limitations when it comes to communicating the quality policy in a way that takes into consideration the information needs of different stakeholders. Information about quality management could be communicated more effectively particularly to students and international members of the university community.

Based on the interviews, students mostly use and rely on email, news feeds and the Tuudo student mobile application rather than accessing information through the Notio intranet. The interviews revealed that students feel that Notio does not contain much information useful for them in managing their studies and they find it to be data-heavy and generally not designed in a manner that caters to their needs. As it is now, the intranet is used mainly by student representatives; for others accessing information on the Tuudo mobile application is easier and more convenient.

However, as the mobile application is mainly an information channel for study-related matters, this raises the question about how ordinary students receive relevant information regarding quality management.

The availability of information in English to international staff and students also needs improvement.

Although international students and staff feel that the university has improved substantially in this regard, the practice of providing information in English still varies across units and faculties.

Generally, international students and staff feel that most quality-related communication takes place in Finnish. The details of information can often be lost when, for example, a long email or a section on the university website are translated into English only with a few lines. The audit team sees that such fragmentation in communication can negatively influence the feeling of integration into the university community. Thus, it is important that the university continues to progress in providing more complete and easily accessible information to the international staff and students.

(22)

3.3 Link between the quality policy and the institution’s overall strategy

As part of reforming the quality system (see Chapter 5.1), the university has demonstrated commitment in updating and developing the quality policy in a manner that properly reflects its revised strategy. The previous Model for Operational Development has been updated and developed into a more recent, comprehensive and elaborate quality policy through years of internal processes and self-evaluations. The objectives and principles of the quality policy are linked to the university’s overall strategy and it provides a clear direction for the quality management system.

The quality system is also firmly linked to strategic and operations management, and in so doing supports the implementation of the university strategy (see Chapter 4). The quality policy and the quality management system promote fluency and high-quality undertaking of operations in the five key strategic focus areas identified in the university’s new strategy, effective from 2016 to 2020. The quality policy puts forward sets of indicators that help the university assess performance in education, research, interaction with society and other aspects of organisational activities.

The audit interviews held with various groups indicated that the pursuit of high quality in all areas of operations constitutes the core value of the quality policy and the fundamental principle underpinning the quality management system and implementation of the university strategy.

The audit team commends the emphasis given at the university to developing a comprehensive, informative and strategically embedded quality policy.

The link between the quality system and the university strategy could be further reinforced by harmonising the manner in which the quality system supports strategic objectives across the core missions and the organisation. For instance, making the application of the PDCA cycle more visible and consistent (see Chapter 4), as well as enhancing the support of the system for developing societal interaction (see Chapter 6.4) could contribute in this regard.

(23)

4 Quality system’s link with strategic management

The quality system’s link with strategic management is very clear and observable. Despite the many changes the University of Oulu has gone through over the last few years, the university’s strategic operations management and quality system operation form an integral whole. Information produced by the quality system serves the needs of strategic management and decision-making. Various channels exist to communicate the information and the information is put to use systematically in the annual planning process. The system works evenly across different organisational levels although more coordination of development activities is needed. Specific responsibilities are executed with commitment and the general principle is that every community member participates in quality work.

The quality system’s link with strategic management is at a developing stage.

4.1 Information produced by the quality system for strategic management

The University of Oulu adopted its current strategy for 2016–2020, Science with Arctic Attitude, following a strategic planning process that had involved the university community widely. The university’s quality policy aims to support strategy implementation by producing information for the needs of strategic and operations management. During the audit visit, the team found compelling evidence that the university’s strategic operations management and quality system operation combine to form an integral whole.

The main tool for strategy implementation is the annual planning process that is supported by the annual calendar of the University’s operations management (Figure 3). Operational steering comprises steering by the Ministry of Education and Culture and the internal steering within the university. The University’s internal system includes the long-term strategy, associated action plans, annual operational plans and budgets for the next year as well as follow-up systems. Top management annually evaluates the strategy and identifies any change needs. The results of the evaluation are discussed in connection with the Board’s annual strategy review. In autumn, the

(24)

units prepare proposals for their unit-specific action and financial plans. In internal target and performance negotiations, the Rector and the Vice Rectors agree with the units on the objectives for the next few years, as well as on funding. The units report on the achievement of the agreed objectives in their interim and annual financial statements.

FIGURE 3. The Annual Calendar of the University as presented in the quality manual.

A key tool of the quality system to support the concrete expression of strategies and targets is the web-based TATU system. The TATU system is used for the internal target and performance negotiations and supports operational planning and budgeting from the unit level to the university level. The audit team holds the internal target and performance negotiations as a strength of the way the quality system integrates with strategic management. The process has been used mainly for quantitative targeting, but from 2016 onwards, the university has also integrated quality targets into the process and the TATU system. Although quality targets have now been introduced to the annual planning process, some of the interviewed staff were worried that because all key performance indicators are quantitative and follow directly from the Ministry’s funding indicators, there is a concern that this does not encourage working towards quality evenly in all fields of the university. However, the different kinds of internal and external evaluations (see chapter 5.1) have been an important source for also obtaining qualitative information that has been used effectively to improve both operations and quality management.

(25)

To communicate the information that the quality system produces in an accessible way, the university has a web-based portal called SISU. It collects different indicator reports on education, research, personnel and finances in one place and provides a dashboard view on the key indicators.

Based on the interviews, the SISU portal facilitates the use of information for staff holding managerial positions and the university is able to learn how effectively its core operations are running with the help of the system. However, the SISU is not used very much by other members of the community and the use of the portal is limited also because it currently provides reports only in Finnish.

On the whole, the information communication channels seemed fragmented. Based on the interviews, staff members get information via email and from the Notio, SISU and TATU systems, and students mentioned email, the Tuudo mobile application, and the WebOodi and Moodle learning management environments as their information sources. The self-evaluation report also pointed to limitations in the usability of data at research unit levels. The audit team sees that the information needs of users at different levels of the organisation should be addressed more effectively and the university should consider developing a more inclusive and integrated internal communications system to enhance coherent information sharing and dialogue.

The interviews indicated that the university strives to improve the methods of following activities and to enable stronger use of indicator and monitoring data. In doing so, the university works towards reinforcing the link between the quality system and strategic management as well as supporting operations management. One step towards this is the new business intelligence system that the university is planning to introduce to replace the SISU portal. The new system will provide timely information more automatically and allow for better analysis of the data. When developing the new system, the audit team encourages the University to improve the usability of the data and find better ways of balancing the volume of data and the ways in which it is presented.

The use of follow-up data in strategic management could be improved, especially by expanding the indicators to cover the strategic objectives more systematically and to help ensure that the operations of the university are well-aligned with the key focus areas of the strategy. It is also important to make sure that processes are not overloaded with information and that sufficient room exists for actions and follow-up.

In addition to the information systems, material about the operation and objectives of the quality system are communicated in university and unit level bodies, as well as in faculty or degree programme level staff meetings. Based on the interviews, the information flow through the line organisation works efficiently. However, information sharing between faculties and units of the university is an area in need of development. The university could do more work in actively enhancing the visibility of institutional expertise and major ongoing activities both internally and externally. New initiatives in this direction – that were positively highlighted in the interviews – include Deans’ meetings, where the Deans from all faculties meet to discuss common issues, the professor forum and the research unit leader forum, which are organised by the HR department twice a year and which serve to build joint awareness of where the university is as an organisation.

(26)

4.2 Functioning of the quality system at different organisational levels and units

The defined management responsibilities, principles and practices create a frame of reference for management that covers the entire organisation. The general principle of quality work is that every staff member and student participates in quality work while top management has the overseeing role over the quality management (see also 3.1) as a whole. Although the positioning of top management at the vanguard of quality work is commendable, the audit team would like to encourage the university to consider balancing the weights between strategic leadership and ensuring openness to the wider community. Based on the interviews, the management of the quality system is in practice rather top-heavy, where staff in management positions tend to launch and decide the majority of initiatives and processes. The audit team sees that engagement at grassroots level could be strengthened (see also 5.1). This holds also for students, whose involvement in quality management currently happens mostly through feedback and student representative roles.

One key issue that became evident from the self-evaluation report and the interview sessions during the audit visit is the need to clearly understand and address the challenges the University may encounter with the integrated quality system without a separate unit for quality management.

These challenges broadly relate to issues of comparability and coordination. The understanding and implementation of quality and quality management were found to be somewhat inconsistent across faculties and education and research units. The audit team recommends that now, having completed the biggest organisational changes, the university could reinforce its efforts towards making the quality system more systematic, without necessarily having to harmonise every procedure and process. The audit team encourages the university to consider improving the coordination of quality management activities and procedures across units and levels of the university, for example, by strengthening network-based coordination. A healthy balance should be ensured between quality management procedures that need to be comparable throughout the university and those over which units may have the flexibility to tailor to their specific needs, in order for the system to function effectively without having a separate unit responsible for it.

However, overall, the audit team was impressed that despite the many changes the university has gone through during the last few years, it has been able to keep the quality system running as an integral whole.

4.3 Quality culture

In 2010, the new Universities Act stressed universities’ autonomy, and the strategy became a very important guiding force. That has meant more strategic steering, management system development and information- based planning and follow-up practices. The management responsibilities, principles and practices contained in the quality system create a frame of reference for management that covers the entire organisation. Based on the audit visit, the defined division of responsibility works mostly effectively, and those holding specific responsibilities are committed to the tasks.

The audit team sees room for improvement in expressing individual actors’ responsibilities and

(27)

the division of responsibilities between different actors in more concrete terms (see also chapter 3.1). For example, the interviews proved that the staff members in the new middle-management positions execute their responsibilities with high motivation, but attention should be paid to making their workload more balanced and predictable (see also 5.2).

Quality management is strongly integrated in the university’s operational culture. The University has a long history in quality work, since the early 1970s, and the current model for quality management is the result of continuous development. Based on the audit interviews, there is a strong commitment across the university to continuously improve the quality system and operations, which is in line with the PDCA principle laid out in the quality policy (see chapter 3.1). The university’s quality culture is also characterised by general awareness of the strategy, which means that activities can be examined against the backdrop of objectives set in the strategy.

The university’s strategic management and operations management combine with the quality system. This process emphasises the significance of setting targets, monitoring and improvement based on the monitoring. The audit interviews suggested that the implementation of the PDCA framework of quality management could be more visible and systematic throughout the university.

The framework is currently not that well internalised, which means that in practice the staff often implement the PDCA cycle without being conscious of it. This elevates the risk of inconsistent application of the quality policy. The team also encourages the University to consider embedding risk analysis more into the PDCA process, in order to be conscious of the intended and unintended consequences of decisions and actions.

The objectives set out by the university in its strategy form the shared goals of the entire university community and in the annual performance appraisal discussions the goals are also set at the employee level. The university has a well-established practice of holding face-to-face discussions with employees. The audit team was impressed by the fact that most of the employees have goals of their own and 99% of the employees have had performance appraisal discussions during this year, which also serves as evidence of the commitment of the different actors. The role of the annual discussions in quality management and in advancing the quality culture will strengthen further in the future, as following the move to include quality goals for the units, the university aims to also include quality-related goals for all employees.

Based on the interviews, stronger communication with students regarding quality stands out as one factor that could still improve the university’s quality culture. The good practice of teacher to student tutoring (see chapter 6.1) could serve as one channel providing it.

(28)

5

Development of the quality system

The University of Oulu has introduced a variety of methods by which to evaluate and develop its quality management system. The quality system produces information through which the university is able to identify strengths and areas in need of development in its quality system; these in turn are used for developing quality management procedures. A culture of self-assessment supports the development work. The development of the quality system after the previous audit has been moderately systematic and strategically-led.

The development of the quality system is at a developing stage.

5.1 Procedures for developing the quality system

The University of Oulu has a quality system in which the responsibilities for quality management are integrated with everyday work, rather than a separate unit primarily tasked for it. The University Executive Group actively engages in leading and supervising the quality management system.

The main tools that the university uses to assess, monitor and follow the development of the quality system are feedback practices, internal evaluations and external quality audits conducted by FINEEC. Since the quality system is closely integrated into everyday work, its development has been strongly linked to other developments in the university’s management and operations management systems.

The audit interviews indicated that a strong culture of collecting feedback exists in the university.

The active feedback practices and internal dialogue support the development of the quality system.

The feedback and assessment practices have been systematised, and this has played a significant role in improving the quality system, which the audit team commends. Together with the feedback, the tools of the quality system that support strategic and operations management (see chapter 4.1) help management also in overseeing the quality system and in identifying strengths and areas in need of development. The developments in the operative information system, in the student feedback system, and structural restructurings are examples of this in practice.

(29)

The awareness of developments outside each staff members’ own units was, however, found to be inadequate during the audit. Developing mechanisms for identifying, selecting, and disseminating good practices across the university would in the audit team’s view be crucial to exploit the considerable potential the university currently has for internal learning. While some limited discussion platforms, such as the Faculty Education Committee and university level education management group exist to share information across faculties, the audit team encourages the university to consider setting up more systematic and wider mechanisms for sharing good practices.

Based on the audit material and interviews, the university has a commendable culture of self- assessment that enables frequent internal reviews, self-evaluations, assessments and reforms.

The self-evaluation report, for instance, mentions those conducted on the activities of the Board, degree programmes, curriculum, study processes, research and IT administration services as well as the research assessment exercises. These institutional evaluations are often linked to reforms and activities aimed at improving the methods and standards of operations and can focus on assessing the quality system in particular or more generally monitoring overall institutional operations.

The self-evaluations conducted in 2007, 2013 and 2017 have been used to considerably improve the quality system and corresponding operational instruments. As an example of a major development, it is through such a process that the university shifted from working with a separate unit responsible for quality management to integrating the responsibilities for quality management into everyday work. The development of the Model of Operational Development, quality policy, division of quality management responsibilities, process descriptions, operations instructions and remodelling of several other managerial and operational components have been the product of such evaluations.

The existing internal systems of quality management also support external evaluations, including quality audits carried out by FINEEC. The university has shown an impressive ability to encourage self-evaluation; most of the areas requiring further review and development were also identified in the self-evaluation report.

In an effort to further develop the quality system, the university’s plans include improving the coherence of the quality system, strengthening communication and coordination, and systematising the implementation of feedback collected from stakeholders. Amongst these, the audit team identifies the fragmentation of quality management efforts and absence of robust coordination as a key challenge. This is because the existing system can be vulnerable to disruptions in the cycle of continuous development because responsibilities for undertaking quality management activities are dispersed across various individuals and units. The audit team sees that it is particularly important to systematically coordinate the otherwise fragmented and dispersed responsibilities for quality management.

Coordination is also important with respect to the quality system as a whole. Although there is clear evidence of successful development, the audit team sees that the system as an entity has not been developed as systematically as it could have been. Development work could be further strengthened by defining an overall process encompassing the individual procedures that the university uses for developing the system.

(30)

Based on the audit material, the initiation, reforming and main responsibilities for developing the quality system largely concentrate around top management. The team sees it as challenging to effectively embed quality management at individual levels so that people across all levels of the university’s hierarchy can fully participate and influence the process of developing the quality system and the university’s outcomes.

5.2 Development work after the previous audit

The main recommendations of the previous audit were related to the risk of under-utilisation of the large volume of information, to the updating of the quality documentation and to having external stakeholders engaging more directly in the quality system. Based on the audit material and the interviews, the audit team sees that the development of the quality system after the previous audit has been moderately systematic and strategically-led by the Executive Group. There is also a shared understanding among the university community that the system has been improved substantially and that it functions better than in the past.

After the audit, the university has been actively working on a wide range of development activities such as carrying out self-evaluations, developing the organisational structure of the university, developing information systems and indicators, and improving information based planning practices and systematising documentation practices. A key development activity has been the continuous improvement of the quality system itself. The self-evaluations conducted on the quality system in 2013 and 2017 have been instrumental in continuously developing the quality system as discussed above in chapter 5.1. In this regard, the development of the Model for Operative Development and the formulation of a comprehensive quality policy and other supportive regulative tools for facilitating quality work are good examples of the extent to which this objective is being accomplished. Efforts have also been made to encourage the active participation of stakeholders, particularly in implementing initiatives and reforms. There are now mechanisms in place to engage internal and external stakeholders in the quality system.

Since the university has introduced several major changes over the past few years, it is the audit team’s opinion that it is now time to pay attention to ensuring stability and to carefully examine the impacts that have emerged from these activities. It is important to conserve the energy, enthusiasm and well-being of all the staff. The audit team highlights the importance of constant vigilance with respect to the workload required for carrying out quality management responsibilities as a result of having an integrated quality system model. Based on the interviews, introducing several changes within a short period has increased the workload of staff who have administrative responsibilities in addition to their teaching and research responsibilities. The audit team suggests that more could be done in making the workload predictable and putting in place mechanisms that provide sufficient support during, for instance, structural and operational reorganisations or internal and external evaluations. The annual calendar of operations could play an important role in this regard.

(31)

6 Quality management of the institution’s core duties

6.1 Degree education

The quality management procedures for degree education are functional with a very good example of implementation of a plan-do-check-act quality cycle. The quality system produces relevant information that is used for developing the degree education but the use of information is not as systematic when corrective actions are planned and implemented. The internal and external stakeholders are properly involved in quality management while the involvement of the external stakeholders remains informal. The quality management of key support services requires further attention in order to develop a systematic and coordinated approach.

The quality management of degree education is at a developing stage.

Functioning of the quality management procedures

The University of Oulu has set out its objectives in line with those of the Ministry of Education. The university provides graduates, post-graduate and doctoral programmes, as well as professional further, continuing and open education, across nine fields. The fields of education are aligned with areas of research excellence, described by the university as being an integral part of its scientific activity.

The University of Oulu has identified six strategic objectives for the development of degree education during 2016–2020:

1. Improving the attraction of the educational offer and student engagement 2. Promoting smooth studying processes

3. Modern, digitised learning environments and pedagogic development 4. International studying experience

5. Structure supporting cooperation with working life 6. Systematic doctoral training

(32)

At the strategic level, the annual planning and negotiation process, supported by the TATU system (see chapter 4.1), is the main quality management tools that translates strategic goals to faculty and unit level targets, which are then followed up during the interim reporting and in the next annual negotiations. Based on the interviews, the strategic objectives are specific, clear, relevant and timely but the audit team suggests that to be fully operational, these objectives could be better aligned with the targeted performance indicators thus making implementation and monitoring easier.

The Vice Rector of Education has overall responsibility for the degree education’s quality management; Education Deans have a similar responsibility at the faculty level. After the change to a research-unit based organisation, the educational programmes do not form their own units but operate in a matrix organisation model. To manage this structure, the University has recently appointed programme directors to be in charge of the degree programmes, and who must negotiate the allocation of the teaching loads with the research units. This system is quite recent, and the university has not yet thoroughly analysed the consequences of this new structure. The audit team regards the appointment of programme directors to help strengthen programme management as a good practice. However, based on the interviews, the programme directors should still be better empowered in their positions in order to ensure smoother delivery of the programmes. The programme directors are often only spending a portion of their time (typically 25%, sometimes up to 50%) in this role while a significant portion of the quality workload falls on this role. This creates a tension between responsibilities and workload. Further attention is therefore required and the creation or development of new support services, or increasing the time allocated for the Program Directors´ tasks could be considered.

Degree programme committees are in charge of designing the curricula and the learning outcomes.

The committees are also in charge of monitoring the implementation by collecting data and conducting annual reviews. During the audit visit, the team received evidence of how these reviews lead to improvement in the degree programmes. Based on that evidence, the quality management procedures for degree education seem to be functioning very well for bachelor and master programmes. The PDCA cycles are generally well implemented and managed at the appropriate level, with a clear definition of roles and responsibilities of the different actors.

The University of Oulu is systematically collecting information about the degree education:

progression and completion, international students, share of students completing 55 ECTS, student feedback, employability surveys, etc. The information is easily available in the SISU portal.

However, during the interviews, the audit team observed variable practices in analysing this data or using them to drive the necessary changes.

Students feel that there are sufficient formal and informal platforms for providing feedback.

Students have access to feedback surveys available at course, programme, university and national levels. The audit team also confirmed that students feel that their feedback is sufficiently taken into consideration and followed by improvement measures. Among others, examples of curriculum revision and the introduction of so-called working life learning, reorganisation of courses in doctoral programmes and widening the accessibility of university facilities to students were given in the interviews. The audit team views the encouragement of staff, students and external stakeholders to provide feedback to improve the quality of degree programmes as a strength. The

(33)

university has managed to create a supportive environment that encourages open dialogue and the flow of feedback. Students feel that they can influence what goes on in their programmes, although at the course level the impact of student feedback could still be improved as based on the interviews, the implementation of the feedback at present can considerably depend on the individual teacher’s motivation, and varies across the university. While the collection of student feedback is systematic, the quality of the surveys could be improved, particularly the course feedback survey where the form in the audit team’s opinion does not meaningfully cover all the aspects related to teaching and the learning environment.

The promotion of smooth studies is shared by the university community. The tutoring system for students, including teacher and peer tutors, is working well and enables individual support for students and their individual study plans. There are also one-to-one development discussions on the progress of individual students and staff towards their personal development plans. Efforts have recently been made to tackle the high number of students who do not acquire 55 ECTS per year. However, the audit team observed often some acceptance of current performance levels, and that the current progression and level of completion receives insufficient attention. Only occasionally, a clear and systematic analysis is performed to understand why students are unable to reach 55 ECTS. The audit team recommends that the university addresses the matter with a systematic action plan. The plan should take into account the many opportunities for improvement that exist, depending on each particular programme, in terms of developing an enhanced student- centred learning experience for all students. It could also serve in making the student learning experience more consistent across faculties and academic units.

Regarding doctoral education, the University of Oulu has recently significantly re-organised its university-level graduate school (UniOGS) with the aim of better harmonising doctoral training and offering modules covering transversal skills or competences. The audit team holds that the creation of a single doctoral school for the whole university is a significant achievement and is already enhancing the quality and consistency of the student experience. The quality management system of the doctoral studies is therefore improving but is still in a transitional situation. It requires further development in terms of stating the learning objectives and outcomes of each doctoral programme, not limited to the transversal 20 ECTS modules that are coordinated by the UniOGS. The audit team recommends that the responsibilities of UniOGS for the quality management of doctoral education should be strengthened to include the collection and analysis of information about progression, completion and employability.

The attractiveness of the educational offer is a concern for the Oulu region as a whole. To address these challenges, the university has intensified their information sessions and developed new activities particularly towards primary and secondary schools. Based on the interviews, these actions are not yet fully planned and developed and further coordination work will be necessary when they require the commitment of several key support services. The University may also consider increasing the role of external stakeholders in this regard as they share this very important concern.

A clear strength is how the university has developed policies and conditions to strengthen internationalisation, including the recruitment of staff and students, mobility, research collaboration and assessment. Arrangements allow students to easily access all information concerning mobility,

(34)

and to systematically transfer credits. In order to further develop their international attractiveness, the university should continue developing more systematic communication in English aimed at international students and staff members (see also 3.2). A systematic approach should seek to ensure a more even quality level when, for example, the same programme is delivered both in Finnish and in English. Appropriate English training should be made available to faculty members and staff when necessary.

Participation in quality work

The formal involvement of staff in university, faculty and programme level committees and boards is evident. The committees and boards are operational and committed to the quality work. In addition to the official bodies, examples of joint development work done by teachers within a faculty were given in the interviews. Additionally, individual teachers have autonomy and responsibility for quality in developing their own teaching.

Students have active representatives in the committees and boards at all levels of the organisation.

Beyond the involvement of the student representatives, however, the students’ role is limited primarily to giving feedback through the various feedback systems. While the audit team finds the feedback culture of the university a clear strength, the team suggests that the university identify more appropriate and useful channels of communications to ensure students are active throughout different phases of the PDCA cycle.

The participation of external stakeholders is visible, but it relies too much on informal contacts.

The audit team recommends that their involvement is developed to be more systematic and significant in its influence and impact.

In order to ensure that the quality culture is more motivating to students and external stakeholders, the audit team suggests the university should communicate the actions and impacts more systematically. Making analysis reports and outcomes easily available is especially pertinent in cases where students or external stakeholders are surveyed.

Quality management of key support services

The university support services have recently been restructured. These changes have met with broad support, and there is a high level of satisfaction across the university community. Based on the interviews, further development is proposed, including improving the services from the user’s perspective and developing user feedback channels. The feedback systems serve as a useful tool for quality management in support services, but they require better integration and coordination in the future. Likewise, the development and implementation of common principles should be addressed. However, the university is well aware of these needs and is planning actions for coordinated development of the support services.

(35)

6.2 Samples of degree education

6.2.1 Bachelor’s degree in Teaching and Education and Master’s degrees in Primary Teacher Education and in Early Childhood Education

The University of Oulu has a long history of providing degree programmes in Teaching and Education.

As such, its position is well regarded throughout the community, and it attracts students from around Finland and internationally. There are well-defined quality management procedures but they are not fully functional and do not adequately keep the curriculum and pedagogic approaches in line with international research and student feedback. Staff, students and external stakeholders have an active role in the quality work of the programme but evidence is missing of a clear enhancement effect of their participation.

The quality management of the Bachelor’s degree in Teaching and Education and Master’s degrees in Primary Teacher Education and in Early Childhood Education is at an emerging stage.

The University of Oulu offers a Bachelor’s degree and two Master’s degrees in education, focusing on early childhood education and primary teacher education, which provide teachers mainly for northern Finland. The programmes are situated within the Faculty of Education, which is one of the eight faculties of the University of Oulu. Both Master’s programmes are connected via the same bachelor’s degree programme. At Masters’ level, students can choose special education, education/

early childhood, primary teaching education or music education. In addition, there are two other Masters’ programmes (music and special education) and two international Master’s programmes (Education and Globalisation, and Learning, Education and Technology) which are not discussed under this audit target. The programme structure of the faculty is presented in Figure 4.

FIGURE 4. Structure of the degree programmes within the Faculty of Education.

AUDIT OF THE UNIVERSITY OF OULU 2018

AUDIT OF THE UNIVERSITY OF OULU 2018

Ellen Hazelkorn Rediet Abebe

Heidi Ahokallio-Leppälä Philippe Bouillard

Kati Korhonen-Yrjänheikki Touko Apajalahti

Anni Tuurnas

AUDIT OF THE UNIVERSITY OF OULU 2018

Ellen Hazelkorn Rediet Abebe Heidi Ahokallio-Leppälä Philippe Bouillard Kati Korhonen-Yrjänheikki Touko Apajalahti Anni Tuurnas

Abstract

Tiivistelmä

Sammandrag

Contents

Audit targets 1

and process

1.1 Audit targets

1.2 Implementation of the audit

2 The organisation of the University of Oulu

3

The quality policy

3.1 Rationale, objectives and division of responsibility

3.2 Communication of the quality policy

3.3 Link between the quality policy and the institution’s overall strategy

4 Quality system’s link with strategic management

4.1 Information produced by the quality system for strategic management

4.2 Functioning of the quality system at different organisational levels and units

4.3 Quality culture

5

Development of the quality system

5.1 Procedures for developing the quality system

5.2 Development work after the previous audit

6 Quality management of the institution’s core duties

6.1 Degree education

6.2 Samples of degree education