Member survey

The author decided to conduct an online survey amongst users of the Alumniportal Deutschland. Online surveys can make experiences by respondents explicit and are an effective tool to collect information from a broad population (2007a). Surveys can produce general data and information that corresponds with the views of the majority of a population (Routio, 2007a). For Tuckman and Harper (2012: 244), “survey research is a useful tool when researchers wish to solicit the beliefs and opinions of large groups of people”, as they allow investigators “to measure what someone knows (knowledge or information), what someone likes and dislikes (values and preferences), and what someone thinks (attitudes and beliefs).” Compared to other methods, online surveys allow including a larger group of users and gaining an overview and initial understanding of this user group (Routio, 2007a).

The member survey that was conducted in the course of the thesis aimed at activating respondents to tell about their online learning behaviour and their experiences with digital learning offers of the Alumniportal Deutschland as well as other providers. Also, the author wanted to generate insights around respondent’s personal views to identify possible improvement areas within the existing service as well as user needs and expectations regarding future services. One goal in this context was to find out which types of digital learning offers on the Alumniportal alumni would be likely to attend, enjoy and find valuable. The survey also aimed at creating insights into users’ online habits and digital skills. Furthermore, the author wanted to acquire knowledge on aspects like users’ learning environment as well as technical and other influencing factors (Nordin et al., 2010: 133).

3.3.1 Data acquisition: survey planning and design

The online survey was planned as a standardized questionnaire consisting of five sections covering the following aspects:

(1) Experiences with learning offers on the Alumniportal Deutschland

(2) Online habits and experiences with offers for digital lifelong learning by other providers

(3) Expectations and motivation regarding digital learning on the Alumniportal Deutschland

(4) Technical requirements

(5) Optional information (demographic information and willingness to further contribute to the learning offers on the Alumniportal in the future)

The maximum 40 questions²³ were formulated based on insights obtained by the literature review. Most questions were designed as single or multi-select questions as well as scaled options from which participants could select the most appropriate alternative. Questions, that asked participants to rate certain aspects of the learning offers or to express their opinions and attitudes were

23 Some questions in the survey were based on conditions, meaning that they were only shown to participants that had answered the related preceding question in a certain way. For example were participants only asked how they rate web seminars on the portal if they had indicated that they had attended one of the portal’s web seminars. Hence, the total number of survey questions varied from 31 to 40 questions.

mostly created using a Likert scale (Routio, 2007b). The survey also included a few open questions to acquire qualitative data based on respondents’

reflections, opinions and ideas.

The planning of the survey included setting up a data analysis plan, which serves as a road map for organizing and analysing the survey data. It organizes the data analysis by attributing specific survey questions to the research questions the survey is aiming to answer. The table below shows the attribution of research and survey questions in the data analysis plan:

TABLE 1. Data analysis plan for the member survey

Research questions Survey questions

How is the overall user experience with learning offers

on the Alumniportal Deutschland so far? Q1 to Q11 What are positive aspects as well as possible pain

points within the existing service? Q3 to Q9, Q11 What are alumni’s learning objectives and what is their

motivation for participation in online educational offers? Q15 to Q17 What are alumni’s need and expectations regarding

online learning offers? Q17 to Q25

How does alumni’s online behaviour look like and how

are their overall digital skills? Q12 to Q13 What other platforms and formats for online LLL do

alumni use? Q13 to Q16

How do the learning context and technical

requirements of alumni look like? Q26 to Q28

Questions 29 to 40 ask for optional information, comprising demographic data and the willingness of participants to further contribute to the learning offers on the Alumniportal Deutschland in the future (e.g. by participating in the co-design workshop). The data generated by these questions also serve as a basis for alumni segmentation in the course of the thesis process. The survey form can be found in appendix 2.

3.3.2 Realization of member survey

The member survey was designed as a standardized questionnaire, which was implemented as a self-administered e-form on the platform LimeSurvey. The

link to the survey was distributed online to members of the Alumniportal Deutschland by posts in the Alumniportal community (around 4,500 members) and the portals social media channels (Facebook, around 113,000 followers and Twitter, around 8,600 followers) as well as by a mailing to subscribers of the portal’s newsletter (around 136,000 subscribers). By distributing the link to the online survey through different channels and making it available online for three weeks, different units of the population were given equal opportunities for participation. Furthermore, the questionnaire was made available in the two operating languages of the portal – English and German – to enable as many individuals within the population as possible to participate.

The introductory text to the survey asked readers to participate in the questionnaire, outlined the purpose of the research and gave background information on the thesis project. Furthermore, it contained a privacy notice and relevant data protection information. Two weeks after the initial announcement of the survey, reminders were sent and posted via the above-mentioned channels. Those who followed the hyperlink and fully completed the questionnaire were included in the sample.

3.3.3 Data analysis and interpretation

Data analysis and interpretation followed the data analysis plan that was set up before publishing the survey. This plan served as a road map for organizing and analysing the survey data according to the survey objectives and research questions. The responses gathered from the online survey were analysed manually using excel. The author started analysing the survey data by organizing the data and identifying significant data. Data from close-ended questions were converted into numeric values. Here, the author decided to count the frequency of response to each option and to determine percentages and in some cases average and mean to make sense of survey results. The quantified data allowed a better understanding as well as identifying trends in user attitudes and behaviour. Throughout the process, the author paid attention to the quality and statistical significance of data. (How to Analyse Survey Data:

Methods & Examples, n.d.)

In the next step, the author looked at the answers from the few open questions within the survey. Data from open questions usually allow for deeper insights into a topic (Amaresan, 2020) However, as data generated by open questions cannot be quantified, it is harder to analyse. An inductive thematic analysis of the open responses was conducted, which is an “accessible and theoretically flexible approach to analysing qualitative data” (Braun and Clarke, 2006: 77).

For the thematic analysis, the author first familiarized herself with the data of the two open response questions. In the next step, data were coded, by highlighting significant words, sections and phrases in the text. Thereafter, the author looked over the created codes with the aim of identifying patterns and developing broader themes. Codes and themes were reviewed, and adjustments were made where necessary, to ensure that they are “useful and accurate representations of the data” (Caulfield, 2020).

When writing up the data analysis report, the author aimed at visualizing relevant data in charts and tables for further usage and interpretation. When interpreting the survey data, the author was careful of drawing conclusions and tested the emergent understandings of the data by cross-tabulation and searching for alternative explanations (Caulfield, 2020; How to Analyse Survey Data: Methods & Examples, n.d). The most significant findings of the member survey are presented in chapter 4.1.