Problem statement and research questions

The increasing number of students in HEI led to an equivalent increase in the number of teachers, as can be seen in Figure 4, that reflects the HEI teacher: student ratio in Portugal. However, “while an ideal teacher:student ratio is around 1:20 to 1:30,” it is very common in HEIs to have classes of

“around 100 students,” which is not beneficial for learning quality (Chebrolu et al., 2017).

28

Figure 4. The number of students and teachers along the years in

Portuguese higher education institutions (PORDATA––Estatísticas, Gráficos e Indicadores de Municípios, Portugal e Europa, n.d.)

This situation has a strong negative impact on assessment activities, in particular, in courses with practical subjects, where individuals must practice and test their knowledge. In fact, with many students, the assessment becomes more laborious, since these subjects are more difficult to evaluate and are time-consuming, often requiring that the teacher be fully dedicated to one student at a time. The e-assessment procedure currently in use at ISCAP, illustrated in Figure 3, imposes a considerable burden on the teacher, as it requires the production and evaluation of several versions of the tests and organizing the students in shifts, whilst being susceptible to failure and unfairness.

The hypothesis is that it is possible to use e-assessment in more innovative ways so these problems may be overcome. In particular, the author argues that it is possible to establish a continuous e-assessment setting, particularly useful for large courses with crowded classrooms, which delivers reliable and fair evaluation whilst not compromising the learning outcomes established for the course.

With this goal in view, two main research questions were defined to act as drivers for the research work of this dissertation study:

29 RQ1: How can higher education teachers use continuous assessment of practical Information Systems topics in crowded classrooms,

without compromising the learning process and learning outcomes of these students?

RQ1 translates into the conceptual aim of this work, which is

understanding how to implement continuous e-assessment of practical subjects in crowded classrooms efficiently, without compromising the learning process. The investigation led to the development of a new assessment method based on multiple-choice question (MCQ) quizzes supported by a learning management system (LMS).

The choice of an LMS platform was derived from previous research (Babo et al., 2012; Babo & Azevedo, 2012; 2009), which provided a broader view of the use of such platforms in HEI. The selection of Moodle as the LMS to use in this work was grounded in the fact that this LMS is one of the most popular within the higher education landscape. Additionally,

considering ISCAP was already using MCQ supported by Moodle to assess theoretical topics, it seemed a natural step also to use it for practical topics.

However, the research to answer RQ1 concluded that MCQ quizzes could not assess all the skills and competencies expected of the students.

It was necessary to complement them with other assessment approaches, notably problem-based learning (PBL). Douglas et al. (2012) also point to this problem, recommending combining these quizzes with other

assessment approaches. Therefore, it was decided to use PBL by gathering students in groups to solve a proposed problem. It was expected that this method could assist with the assessment related to the skill and

competency acquisition that MCQ quizzes lacked.

While the use of PBL on a group basis allowed solving flaws of MCQ quizzes toward finding an answer to RQ1, it also led to the identification of another problem. Given that most of the students’ work is developed during non-contact hours, that is, outside classes and, thus, without the consistent supervision of teachers, it is not possible to have a good

30

perception of the individual performance of each group member. This posed great difficulty in differentiating students according to their commitment and skills, leading to assigning the same grade to all group members and, thus, to (re-)introducing unfairness in the evaluation process (Figure 5).

Figure 5. Group assessment without individuals’ distinctions

Considering this situation, the second research question was formulated.

RQ2: How can teachers assess individuals working in groups and guarantee that they are assessed according to their performance and contribution to the work developed? How can a software tool assist evaluators with their task of assessing everyone in a workgroup?

The second goal is a natural extension of the first, intending to answer how the assessment of workgroups can be performed in a fair and simple manner. In 2009, Luxton-Reilly expressed the need for the development of peer assessment tools that can be used by various institutions. This study also refers to the need for more usability studies on available tools

(Luxton-Reilly, 2009).

Since this dissertation is contextualized in Portugal, namely a public higher education institution, ISCAP, it is important to note the need for a freeware tool. The tuition fees in Portuguese universities and polytechnics are low in comparison to other countries, where, as reported by DGES website, the maximum value fixed for tuition fees is € 697. Usually, there is

31 hardly financial availability to acquire new softwares. Therefore, to

innovate and bring different technologies to classrooms, lectures many times depend on their own tools, which in some cases can be the use of freeware (Propinas, n.d.).

The work conducted in the context of this dissertation over a period of years allowed the answering of these research questions. The outcomes of such work have been presented in original publications (articles I–VII) and, thus, validated by the relevant research community. Figure 6 illustrates the new e-assessment approach that resulted from the research developed.

Figure 6. New students’ e-assessment approach

To answer the research questions, research work was conducted in successive phases, and the outcomes were presented in articles I–VII.

Articles I to IV result from the research work developed to answer RQ1, whereas articles V to VII present the work developed to answer RQ2.

32