This chapter discusses the data analysis and presents the insights gained from the research and service design methods. Data gathered and processed through the study was carefully analysed. The author aimed at presenting the data clearly and visualizing them with the aid of tables and diagrams to facilitate reader-friendly and coherent writing.
4.1 Member Survey
This chapter presents the data analysis and findings derived from the member survey. It starts with introducing the demographics of participants before presenting the survey findings regarding the seven guiding research questions for the survey. The findings mainly cover four broad themes, including demographics of users, their experiences with digital learning, their objectives and motivations for digital learning, as well as environment issues. The author also aimed to measure responses by selected demographics to better understand the data according to alumni segments. However, cross-tabulation of data based on the demographics origin (differentiation according to five continents), age group (differentiation according to three age groups) as well as gender (differentiation according to three genders) did not reveal significant findings for these alumni segments. Overall, 1,392 responses to the survey were received of which 612 questionnaires were submitted complete and evaluated. The 612 questionnaires that could be analysed equal a completeness rate of roughly 44%. This rate could indicate that the survey was not relevant enough for some participants or too long and comprehensive for others29. 246 participants completed the questionnaire in English, and 366 participants completed the German version.
The final sample group can be considered a self-selecting, representative and convenient subsample regarding the key demographics of the overall population. Statistical analysis reveals that participants of 95 different
29 Before publishing the survey, the researcher did several test runs with team members, which did not provide any indication that the survey was too long.
nationalities took part in the survey, comprising 26% from South America, 25.5% from Europe, 23.4% from Asia, 22.6% from Africa, 2.3% from North America and 0.3% from Oceania. The sample includes 35.1% (215) females, 55.1% (337) males, and 0.1% (5) participants of other gender. 9% (55 participants) did not specify their age. The age distribution shows that most age groups were represented in the survey except for the very young user groups (younger than 20 years). The author assumes, that this age group is usually still enrolled in some kind of formal education and therefore their interest in non-formal LLL learning offers naturally is lower than that of older age groups. The age group 31 to 60 years, people of working age from early to and peak career level, had the strongest representation (71.5%) and can be defined as the key target users of the learning offers on the portal. The complete table showing the age distribution of participants can be found in appendix 6.
The survey sample comprises individuals with different career status: 30.3%
identified as employees; 29.6% indicated that they have an academic position and 13.1% are self-employed. Other career statuses comprise pensioners (9%), university students (5.6%), currently without position (4.7%), graduates (2.2%), students (0.9%), and other (4.8%). Data analysis also shows that the educational level of survey participants is remarkably high: 83.8% of participants indicated that their highest educational level is a Master’s degree, equivalent or higher30. The diagram below shows the top 5 fields of study, research and industries that people assigned themselves to: The largest share (37.2%) work in the field of Education, Science and Research.
30 The top 5 answers regarding participants highest educational degree comprise: PhD 33.1%
(186), Master’s degree 30.1% (169), Bachelor’s degree 11.2% (63), Magister degree 11% (62), and university diploma 7.8% (44).
FIGURE 1. Top5 fields of study or research/ industries of participants
How is the overall user experience with learning offers of the Alumniportal Deutschland so far?
Statistical analysis has shown that participants’ average satisfaction with the digital learning offers on the Alumniportal Deutschland is quite high. For the seven examined learning formats31 it lies at 88.5% (average of positive ratings (excellent or good) for all learning formats and all examined aspects). The survey examined user satisfaction with the attended digital learning offers32 regarding the following aspects: quality of content, expertise of trainer or speaker, provided material, technology or software used, learning success, application or registration process, communication and accessibility of organizers (before, during and after the event), interaction and exchange with trainers or speaker, and interaction and exchange with other participants. The table below shows the average positive rating regarding these aspects (rated as excellent or good by participants) for all examined learning offers. The biggest satisfaction exists regarding the quality of content (96.1%) as well as the expertise of the trainer or speaker (94.4%).
31 The examined formats comprise tutored eLearning courses, Hands-on-Projects, Blended Learning courses, Virtual Coffee Breaks, Alumniportal Labs, web seminars, as well as online self-study courses.
32 Web seminars are the format that most survey participants have attended on the Alumniportal Deutschland (242), followed by online self-study courses (172). The other formats were less frequently used by survey participants: Alumniportal Labs (60), Tutored eLearning courses (51), Virtual Coffee Breaks (43), Blended Learning courses (39), and Hands on Projects (25).
25.1%
The 5 most frequent fields of study or research/
industries of participants
FIGURE 2. Positively rated aspects of examined digital learning offers (rated as excellent or good by participants)
When comparing the examined learning offers against each other, survey data revealed that tutored eLearning courses, Hands-on-Projects and Blended Learning courses received the most positive ratings regarding the examined aspects33. Formats that were rated positively by fewer survey participants are online self-study courses (86%) and web seminars (86.1%).
FIGURE 3. Digital learning offers rated positively regarding the examined aspects (rated as excellent or good by participants)
33 Hands-on-Projects and Blended Learning courses are currently not offered on the Alumniportal Deutschland.
Positively rated aspects of digital learning formats
92,1
Positively rated digital learning formats
The survey also asked participants to rate utilized learning materials34 regarding the aspects quality of content, clarity, design, timeliness, applicability, and learning success. The evaluation of the learning materials by participants was quite positive. For learning materials, participants’ average satisfaction ranks at 88.9% (average of positive ratings (excellent or good) for all learning materials and all aspects). The table below shows the share of people who have rated the respective aspects of the learning materials positively (excellent or good).
TABLE 3. Aspects of digital learning materials rated positively by respondents quality of content 94.3%
clarity 91.5%
design 88.5%
timeliness 87.6%
applicability 87.2%
learning success 84.0%
What are positive aspects as well as possible pain points within the existing service?
Looking at the examined learning formats and materials it was revealed that some aspects were rated more positively than others. Regarding learning formats the perceived quality of content (96.1%) as well as the expertise of speakers and trainers (94.4%) was rated as excellent or good by the largest share of survey participants. Regarding the examined learning materials, the clarity of material (91.5%) was rated especially positive.
Aspects of the examined learning formats that were rated positive by a smaller share of participants (rated as excellent or good by less than 90% of participants) are the application/ registration process (89.3%), the accessibility of and communication with organizers (88.9%), the interaction with trainers or speakers (82.5%), as well as the interaction with other participants of the learning offers (71.6%). As to the learning materials, the aspects of timeliness (87.6%), applicability (87.2%), as well as learning success (84%) received the
34 The learning materials that have been utilized most by survey participants are quizzes (194), eBooks (180) and videos (169).
least positive ratings. It could be worth looking at these aspects in more detail to find out the reasons behind the less positive ratings and finding ways for improving them to increase user satisfaction.
The survey also aimed at assessing the effectiveness of different channels for communicating and marketing the digital learning offers on the Alumniportal Deutschland. 48.1% of participants indicated that they got to know about the attended learning offers by the Alumniportal newsletter, 30.8% have learned about the offers on the Alumniportal website, 6.8% got notified by Social Media posts, 6.6% by friends and acquaintances, 6% by post in the Alumniportal Deutschland Community and 1.7% indicated other sources of information. The table below shows how participants would like to be notified about digital learning offers in the future. The newsletter appears to be a preferred and effective communication channel. The findings also show that users would like to receive personalized information and invitations. This is supported by findings of the literature review: organizations should create personalized and customized offers and marketing activities towards alumni (Barnard et al., 2006;
Trends in Alumni Relations, 2018). In this context, it would be advisable to further explore the communication options the portal’s community offers based on generated user data.
TABLE 4. Preferred ways of being notified about learning offers on the Alumniportal Deutschland in the future
Newsletter 468
Personal invitation to events that suit my preferences 342
Social Media post 161
Sitemap which allows to browse and search for
suitable offers 137
Community post 131
other (email) 12 (10)
What are alumni’s learning objectives and what is their motivation for participation in online educational offers?
When asked about their general learning objectives, as large share of alumni indicated that they wish to gain insights into different topics (510), while an
almost equally large share noted that they wish to specialize in certain topics (486). Another important motivation for participating in the learning offers is exchanging ideas, experiences and knowledge with others (473). While receiving a certificate is a motivation for fewer users to participate in the learning offers (379), it is still important for many to receive some kind of feedback on their learning progress (452).
When asked which topics, knowledge, competencies and skills the Alumniportal’s learning offers should address in the future, the top five response options were ‘Learning and practising German’ (427), ‘Academic and specialist topics’ (328), ‘Digital competences and skills’ (327), ‘Career & employment-related topics (320)’, as well as ’Intercultural communication’ (288). For the most part, this reconfirms the findings of the 2016 member survey, which had shown that a large proportion of participants are interested in the topics
‘German language learning’, ‘science and research’, ‘career tips and application training’, as well as ‘career and management topics’ (Results of the Alumniportal Deutschland Survey 201635, 2016) The table below shows the relevance of different topics and skills to alumni within the scope of the digital learning offers on the Alumniportal Deutschland.
TABLE 5. Preferred topics and skills to be addressed by the digital learning offers on Alumniportal Deutschland in the future
Learning and practising German 427
Academic and specialist topics 328
Digital competences and skills 327
Professional learning (career & employment-related
topics) 320
Intercultural communication 288
Cultural awareness and expression 264
Leadership/ Management Skills 260
Sustainability 238
Soft skills 226
35 Internal source, not publicly available
Entrepreneurship 198
Civic and social issues 197
Capacity building 176
Other 14
What are alumni’s needs and expectations regarding online learning offers?
When asked about specific learning formats that alumni would like to see on the Alumniportal Deutschland in the future, web seminars (404), online self-study courses (347), as well as online conferences (338) received most votes.
Regarding learning materials, educational videos36 (455), eBooks (432) and exercises for practising and testing oneself (352) were the most popular choices of survey participants. The diagrams on favoured future learning formats and materials are displayed in appendix 7. Regarding the frequency with which alumni would like to participate in digital learning offers, there is balance of people of who would like to participate every week and several times a week (29%), 1-2 times a month (35%) and less frequently “from time to time” (35%).
Regarding the duration and amount of time, they would like to spend on digital learning on the Alumniportal, the largest share of survey participants (61.6%) indicated that they would like to invest 1 to 6 hours per month. Among those are participants who are ready to spend 1 to 3 hours on the digital learning offers (32%), as well other who would like to invest 4 to 6 hours per month (29.6%)37.
36 In the open text response, the learning formats and activities that were mentioned most often by participants are videos and audio-visual content (9 mentions). The table with the identified codes and themes can be found in appendix 9.
37 The open text response brought up further comments on factors around time: Participants stated that the provider should consider the duration and timing of learning formats and ensure brevity and briefness of offers, as well as continuity and frequency of content. The table with the identified codes and themes can be found in appendix 9.
FIGURE 4. Frequency with which alumni would like to use digital learning offers on the Alumniportal Deutschland in the future
To find out more about alumni’s needs and expectations regarding digital learning offers, the survey asked participants to rate the importance of twelve aspects of learning offers: quality of content, skill level of teacher or expert, quality and amount of provided resources, exchange with experts/ trainers and other alumni, active practising, receiving an attestation of participation or certificate, discussing diverse opinions, time flexibility, free access, and reputation of the provider. The top5 aspects that were rated as very important or important by most participants are quality of content (591), quality of provided resources (567)38, skill level of teacher or expert (564), time flexibility39 (525) and free access (518). The three aspects that least participants voted to be very important or important are receiving an attestation of participation (361), exchange with other alumni40 (354) as well as receiving a certificate (348).
However, these aspects are still relevant to more than 50% of all survey participants and therefore should not be neglected completely.
38 The open text response included hints on how to further increase the quality of the digital learning offers. Most people noted that the offers have to be innovative and state-of-the-art (5 mentions). The table with the identified codes and themes can be found in appendix 7.
39 That flexibility of digital learning offers is an important factor to alumni is supported by the fact that 520 survey participants (85%) indicated that they fully agree or agree with the statement “I wish to be flexible regarding the time of learning”. 482 participants (79.2%) fully agreed or agreed with the statement “I prefer self-paced learning.”
40 The number of participants, who indicated that they prefer to study and practice by themselves (477; 78%), exceeds the number of those, who indicated that they prefer to study and learn in a group (332; 54.2%).
How often do you want to use digital learning offers on the Alumniportal Deutschland in the future?
FIGURE 5. Aspects that are important to alumni when participating in digital learning activities
How does alumni’s online behaviour look like and how are their overall digital skills?
The survey data shows that a large share of participants is quite active in online activities. The author assumes that those individuals have a certain degree of digital skills. When asked how often participants engage in online learning and teaching activities: 66.9% of participants have stated to engage in online learning often or every day and 42.5% have indicated to engage in online teaching often or every day41. The table below shows the online activities that participants have stated to do often to every day.
TABLE 7. Regular online activities of survey participants
Research activities 71.8%
Learning activities 66.9%
Content creation 48.4%
Teaching activities 42.5%
Online project work/ workshops 36.2%
Commenting 32.6%
41 There might be a non-response bias in the survey data, as people with limited access to the internet or a poor internet connection might not have been able or willing to participate in the survey. These individuals naturally engage in fewer online activities and their digital skills might be lower.
591 567 564 525 518 495 464 452 416 406 361 354 348 1000
200300 400500 600700
What is important to you, when using online learning and training offers? (rated as very important &
important)
Cloud working 32.3%
What other platforms and formats for online LLL do alumni use?
Survey data revealed that alumni like to draw on educational institutions, above all universities (402) but also other educational institutions (244) when looking for learning offers online. Further sources for digital learning that alumni rely on include industrial and commercial providers (173), alumni associations (161), employers (125), MOOCs (120), as well as other sources (not specified) (24).
When asked about concrete platforms that alumni use to browse and search for learning offers, YouTube is by far the most used platform (474). Other popular providers include Coursera (206), LinkedIn Learning (166), Duolingo (148), as well as edX (126). The chart that depicts the top10 mentions of digital learning providers is shown below.
FIGURE 6. Popular providers for digital learning among survey participants
The author decided to include some of these providers that are relevant to users of the Alumniportal in the benchmarking. Regarding offers and formats by other providers that alumni have attended in the past, the top5 formats are web seminars (403), video conferences (345), online self-study courses (294), workshops (236) and large digital conferences with 50+ participants (224).
474
206 166 148 126
85 63 60 44 43
50 0 100 150 200 250 300 350 400 450 500
Which platforms have you used to browse and access digital learning offers before? (10 top answers)
How do the learning context and technical requirements of alumni look like (including possible obstacles and challenges)?
Statistical analysis revealed that the large majority of respondents appear to have the technical requirements to participate in digital learning offers. Survey data showed that most participants access the internet via laptop or notebook.
On second rank, smartphones are the device of choice. This is supported by the findings of the 2020 web monitoring, which showed that in the evaluated period, 53% of site visits for the German version of the website came from a desktop PC, laptop or notebook and 42% of site visits happened by a smartphone.
Tablets accounted for 5% of site visits. (Alumniportal Deutschland Web-Monitoring Januar 2017 bis Mai 2020, 202042). These findings support the ideas that the conceptualization and design of the Alumniportal’s learning offers should take the specific requirements for mobile learning into account.
Regarding participants’ internet access, 74% rate their access to the internet to be very good and state that they have access to the internet whenever they want. Furthermore, the vast majority of participants (96.7%) states their internet connection to be good (fast and constant) or at least okay (working fine most of the time, with some disruptions). The diagrams on the internet access and quality of the internet connection can be found in appendix 8. In this context, there might be a non-response bias, as people with limited access to the internet or a poor internet connection might not have been able or willing to participate in the survey.
Further Findings
Survey participants were invited to leaver further comments in two open response questions. The first open response question asked for further aspects that are important to respondents regarding digital learning. The question generated 45 responses, of which 43 responses were analysed in a thematic analysis. Analysis revealed 13 themes that emerged within the responses. The second open response question, which invited participants to leave comments on anything else they wanted to tell the researcher on the topic at the end of the survey, generated 116 responses, of which 52 responses were related to the topic of the survey and were analysed in a thematic analysis43. Analysis
42 Internal source, not publicly available
43 Other open text responses were not related to the research questions. They comprised feedback on the survey and the thesis project (32 comments, all positive) as well as general
revealed that nine main themes emerged in participants’ responses to this question.
Although the survey was of exploratory nature and the thematic analysis was guided by an inductive perspective, during the analysis process the author discovered that the emerging themes of responses to both questions could be combined and that they could be linked to the four perspectives and design requirements offered by the framework for mobile learning for lifelong learning suggested by Nordin, Embi and Yusuf (2010) which is presented in chapter 2.5.
The table which shows the identified themes and codes and allocates them to the four categories generic mobile environment issues, mobile learning contexts, learning experiences, and learning objectives (Nordin et al., 2010) can be found in appendix 9. The main findings are summarized below.
Generic mobile environment issues
The three themes that emerged regarding generic mobile environment issues comprise issues of information (8 mentions), new user and (indirect) target
The three themes that emerged regarding generic mobile environment issues comprise issues of information (8 mentions), new user and (indirect) target