Summarizing the systematic mapping search results

On the basis of the search results, it can be said that the interest in the field has been steady with some variations. In the early 2005 the field of research had a spike of publications with a steady decline until 2008, after which the number of publications has been rising slowly. Several of the excluded papers discussed CSCL in general without mentioning software engineering education, so the interest in the more general field of CSCL seems to be strong.

The articles were divided into four topics by utilizing first the LDA topic modeling feature in NAILS bibliometric software (Knutas et al., 2015) to establish the categories and initial paper assignment, and then by refining the classifications manually. The four discovered topics can be thematically summarized as interaction analysis (T1), software engineering projects (T2), cognitive studies (T3), and online teamwork case studies (T4). T3 was excluded from this literature review, because cognitive studies are out of the scope of this research.

The four categories can be described quantitatively with the four most common words in each category (Figure 3.1), or qualitatively. Qualitatively described, topic T1 concerns studies in interaction analysis, where the publications present methods of classifying and analysing student interactions, e.g. statistically. Topic T4, which is closed linked with T1, presents case studies where CSCL has been applied to distance or online education. Many of the case studies presented in T4 apply interaction or other statistical analysis methods. Topic T2 includes publications that feature computer-supported collaborative software engineering projects, and also articles about the development of new CSCL tools.

The distribution of LDA topics was visualized with the LDAvis library (Sievert and Shirley, 2014), which shows the relative division of text corpora between each topic, the four most descriptive words for each topic and intertopic distances.

The resulting visualization is presented in Figure 3.1. The relative sizes of the circles represent one interpretation of the prevalence of each topic in the dataset, and the locations represent the intertopic distance calculated with PCA (Principal Component Analysis). The distances are relative in the sense that they cannot be compared to visualizations of other datasets. However, visualization is useful in giving an overview of cluster similarities. For example, it can be seen that according to the LDA analysis, the text corpora of topics T1 and T4 are close enough, so that there is some overlap. Topic T3, which was excluded from the review, is surrounded by a dotted line.

42

3 The state of computer-supported collaborative learning in software engineering education

Figure 3.1: Intertopic distances between LDA-discovered topics

The thematic analysis based on the LDA is a novel contribution in this thesis.

An alternative, qualitative mapping is presented in Publication II. The following paragraphs describe each category in detail and discuss the state of research, except for T3, which has been excluded.

T1. Interaction analysis There were numerous articles that analysed interactions quantitatively, and these studies concentrated either on team behavioral patterns or collaborative communication patterns. These patterns were presented as statistics or social network analysis graphs. Serce et al. (2011) and Swigger et al. (2012a,b) present findings on the behavior of global, distributed student teams. Their results showed that the communication patterns were related to the teams’ communication modes, the nature of the task and the experience level of the leader, and that there was a positive correlation between the communication patterns and project outcomes. One of the first case studies that used the method of applying social network analysis to analyzing CSCL was published by Martínez et al. (2003). Later studies, such as that of Knutas et al. (2013) used this approach to study the communication patterns that occur in CSCL classes and identify patterns that emerge during the progress of the course. Vivian et al. (2013) expand the interaction analysis by coding interactions by types. Additionally, Duque et al.

(2009; 2012) present a design for collaborative interaction analysis and recording system to use in interaction analysis, and Karakostas and Demetriadis (2011) discuss the possibilities of pattern-based adaptive CSCL systems.

The research into interaction analysis has introduced incremental improvements to the methodologies when applying these methods in the research of online communication. However, one weakness when researching collaboration online is the relative lack of automation. Observing and establishing the communication context still requires manpower, though research in automated analysis is ongoing, especially in the field of interaction classification and social network analysis.

T2. Software engineering projects The articles in this category can be divided into case studies on some aspect of collaborative learning in software engineering projects, and papers presenting new collaborative tools for software engineering. The case studies on collaborative learning included studies where software engineering was enhanced through participating in open source projects (Papadopoulos et al., 2013), establishing communities of practice among students and start-up companies in high-tech entrepreneurship (Rohde et al., 2005), and teaching collaborative software development in an environment that considers the social aspects of collaboration with a reputation system (Kilamo et al., 2012). Many of the case studies also involved featuring a new collaborative tool, like using a new online project environment to teach distributed software development (Schümmer et al., 2005), or learning programming teamwork skills in an e-learning platform (Sancho-Thomas et al., 2009).

Articles that concentrated on presenting a new tool for collaboration involved presenting a groupware system for conducting senior projects (Chen and Teng, 2011), using version control in project-based learning (Milentijevic et al., 2008), and presenting a platform that combines several e-learning modalities under a single platform (Cabrera-Lozoya et al., 2012). Tools that involve collaboration directly in creating software included an approach for software design mentoring (Coelho and Murphy, 2007), and platforms for collaborative programming (Duque and Bravo, 2008; Bravo et al., 2013).

The papers presenting the diverse tools and approaches generally reported that they had had beneficial results in the environment the tools and approaches were tested in. In several studies, CSCL tools had affected collaborative communications positively in both online and physical environments. For example, Scheele et al.

(2005) found that using CSCL tools to make lectures interactive increased student engagement and interest and enhanced learning. As regards online communities, Martinez-Mones et al. (2005) found that students learn concepts better on their own, but they are able to generalize collaboratively discussed concepts better. In virtual learning environments, collaboration was shown to increase the learning outcomes (Elmahadi and Osman, 2013). A case study by Shaw (2013) showed that small groups with external support in online collaboration achieved the best learning outcomes.

Many of the papers featured a virtual learning environment, a collaborative communication system or other tools in support of software engineering, while

44

3 The state of computer-supported collaborative learning in software engineering education

some presented tools for collaborative programming. This division indicates that there are two levels in supporting collaboration in software engineering:

collaborative programming and collaboration around other tasks, like problem solving or task negotiation.

T4. Online teamwork case studies Several studies about the implementation of CSCL examined cases where one or several communities used the same collaboration system. The first article found in the search presented strategies for establishment and management, concluding that the establishment process is mostly student-driven (Sheard, 2004). A later article by Serce et al. (2010) presented strategies and guidelines for building effective globally distributed student learning teams. Several articles presented case studies and positive experiences about distributed learning communities in software engineering education in Italy (Coccoli et al., 2011), Europe (Papadopoulos et al., 2013) and Latin America (Giraldo et al., 2010).

Several articles considered the implementation of CSCL in local and global contexts, with overall positive results. These studies reported that computer-supported collaboration can be established either locally or in globally distributed teams, and that initial surveys had positive results. However, the wider the studies were, the less in-depth ones they generally were. When comparing the papers in this category with the ones considering the effects of CSCL, these papers covered more ambitious case studies at the expense of depth. Overall, the studies showing positive experiences of wide collaborative communities were promising.

The basic premise of wide computer-supported collaborative communities was shown to be valid, and in the future in-depth studies could analyze the success factors of different CSCL approaches.