The first major limitation that has to be acknowledged is the number of design iterations. The design process could have benefited from a third design and test iteration in order to integrate the more advanced gamification features introduced in Publication V. However, the iterative design process is still ongoing and the third iteration of the research artifact will be published in a separate article to address this limitation.
The second notable limitation is the scope of the research. While the research process itself was consistent and valid according to the design science research guidelines (Hevner et al., 2004), as presented in Table 5.1, it was limited in scope.
The surveys and requirements gathering were broad in scope, covering several organizations in different countries. The testing itself was narrower in scope due to practical constraints, involving only single classrooms at a time. It can be argued that the requirements gathering and survey results can be generalized to some extent, but the artifact design would need further testing in order to be able to generalize the results.
The other limitations concern issues that tend to arise from a multi-year experimental design that bridges several publications. For example, the survey design between iterations 1 and 2 was regrettably not identical and the questionnaire scales had to be normalized occasionally before presenting them.
While these limitations are worth mentioning, and survey comparison between the two iterations is not as statistically valid as it would be in an optimal situation, they were ultimately no obstacles to presenting or analyzing the research results.
This thesis has utilized empirical research and the design science research method to examine and then improve the state of computer-supported collaboration in the field of software engineering education. This chapter summarizes the contributions, addresses the research questions and outlines aims for future research.
This study offers three major contributions to the research area, each connected to different design science research cycles. The research provides practical value to the application domain by addressing the presented problem in a manner that satisfies the requirements. The research artifact produced by the design cycle can be published to be available for the community. At the same time the publications produced in the rigor cycle add to the scientific knowledge base, extending the current level of knowledge on both theoretical and empirical levels.
This study addresses the research gap by advancing computer-supported collaboration, especially in inter-team collaboration, and advancing the theory of gamification for collaborative environments. It provides new practical knowledge, especially regarding intensive-format courses, and these results can be extended towards capstone-style teamwork courses (Dunlap, 2005) with some more research effort. The practical solutions provide value to the teaching community, and the publications offer empirical evidence and advance in theory to the scientific knowledge base. The following list summarizes the findings of this study:
• The current teamwork patterns in certain types of software engineering courses are limited, especially regarding inter-team collaboration and shared task discovery. Students might work on similar issues without being aware of it. This is something that can be addressed with the use of computer-supported collaboration systems.
• According to systematic mapping studies, there is a research gap in the application of gamification to computer-supported collaboration, and advancing the state of the art has potential for benefits. The systematic mapping study in this research project found that previous studies had established the benefits of CSCL compared to traditional teaching. However, comparing different CSCL methods to each other has not received as much attention.
• In the problem identification phase of this study it was found that the following aspects can be improved in software engineering student collaboration: goal visibility, explicit goal negotiation, publishing current issues, and facilitating help. Improving these aspects has the potential to improve the collaboration process.
• Software engineering students can be encouraged to collaborate online with the application of gamification, and this collaboration has positive
86 6 Conclusion
results for learning goals. This was tested with the first iteration research artifact and the related evaluation case. Similarly, a more advanced version of the gamified collaboration system increased the amount of inter-team collaboration and shared problem solving in a software engineering project course in the second design and evaluation iteration.
• The gamification approach for computer-supported collaboration was developed further by connecting it with the theory of player profiles.
Different types of players respond best to different kinds of rewards, for example simulated social status or additional challenges instead of just an increased score. This study introduced a method for creating gamification profiles from empirical observations in collaborative learning environments.
In short, the study utilized a design science research process to examine computer-supported collaboration in software engineering education environments and then improved it. The major contribution of the study was creating methods to improve and increase inter-student and inter-team collaboration by utilizing gamification and increased task visibility. It also advanced the state of the current theory of adaptive profile-based gamification, and two literature reviews were produced.
6.1 Addressing the research questions
This subchapter provides a summary of in which chapters and how the research questions were addressed.
RQ1a. What collaboration-related communication and activities occur in collaborative software engineering courses?
Students collaborate and interact with each other in software engineering courses on software engineering work and collaborative learning, although the amount of collaboration varies by team, and the patterns of collaboration can be suboptimal (chapter 3.2). Almost all the groups in the observed courses used a significant amount of online resources.
RQ1b. What kind of issues or needs exist in collaborative communication and activities that still need be addressed, in the context of collaborative software engineering courses?
Students do collaborate with others when required, but the patterns of collaboration tend to follow pre-established social connections, and not all groups benefit equally from the collaboration (chapter 3.2). Students might work on similar issues without being aware of it. The main issues that prevented collaboration were the lack of team commitment, non-efficient communication and goal mismatches (chapter 4.1). Some teams only cooperated, despite the encouragement to collaborate.
in software engineering courses with a computer-supported collaborative environment?
Flexible issue-based communication and shared goal setting, combined with a positive feedback loop acknowledging positive actions had a positive impact on collaborative activity. An artifact design for a new collaborative system was presented in chapter 4. Its design was based on cooperative (Johnson and Johnson, 1994) and collaborative (Dillenbourg, 1999b) learning theories, along with the theory of gamification (Deterding et al., 2011).
RQ3. How does a computer-supported collaborative learning environment, with design based on the results of RQ1 and RQ2, affect intra- and interteam student communication and collaboration?
The patterns of collaboration changed positively in regard to inter-team communication and problem-solving, in comparison to baseline collaborative courses (chapter 4.5). The communication features of the collaboration system were perceived to be useful by the students in the test cases (chapter 4.3). However, the impact on intra-team communication was inconclusive and requires more research.