F UTURE WORK - Online course development framework and case: online geographic information syst

7.2 Future work

Since this is a new framework without much testing, it should be tested in multiple courses using different online platforms. In LUT, some existing courses such as the Computer Science 1 and object-oriented programming are using recorded lecture videos, quizzes and online assignments. The courses that are already mostly online and already implement many parts of this framework on their own could be redesigned to use the provided framework and see if the satisfaction increases. This testing would provide valuable results about the framework’s effects.

8 CONCLUSIONS

This thesis research is conducted in Lappeenranta University of Technology as a study to create an online independent learning course on geographical information systems. For the course, a framework is created to be able to create an effective online learning environment. There are no existing geographical information system courses in Lappeenranta University of Technology, so the whole course had to be created from scratch. Most often geographical information system courses are taught in geography-oriented universities or programs and it is rarer to have such course in a technical university.

This thesis addresses the gap in design frameworks for designing online courses. There are no practical frameworks for creating online courses. Most of the existing frameworks are either too abstract or list pedagogical attributes, such as motivation, that should be implemented without explaining how to do it. The framework in this thesis tackles the problem of ambiguity and tries to explain how different pedagogical attributes can be implemented in an online environment, what activities an online course should implement and how to connect the different activities within a course together.

Some courses in LUT already make use of things described in the framework, such as having short videos and questions in-between videos. One such course was well evaluated during the last semester. That course got an overall grade of 3.83 out of 5 in motivation from participants. The course was to be studied independently except for the voluntary exercises where students could receive guidance. In these exercises, there usually were no students present. Those who came usually attended because they could not install the software at home. Overall, more than 90 % of the students did everything at home.

The results from the testing and survey are promising. In the survey most agreed with the ideas that are implemented in the framework and most problems from testing are related to the content rather than the framework itself. These problems are easily fixed for the first real testing of the course. Overall, this study can be seen as successful.

REFERENCES

1. Rukobo EZ, Paul A, Shrem J (2012) The Guidebook for standards-aligned online course evaluation and creation. In: Educ. E-Learn. Innov. ICEELI 2012 Int. Conf.

On. IEEE, pp 1–4

2. Swan K, Matthews D, Bogle L, Boles E, Day S (2012) Linking online course design and implementation to learning outcomes: A design experiment. Internet High Educ 15:81–88

3. Higher Ed Course Design Rubric | Quality Matters.

https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric.

Accessed 2 Nov 2017

4. Silvén M (1991) Kohti itseohjautuvaa opiskelutaitoa. Turun yliopisto, täydennyskoulutuskeskus

5. Ambrose SA, Bridges MW, DiPietro M, Lovett MC, Norman MK, Mayer RE (2010) How Learning Works: Seven Research-Based Principles for Smart Teaching, 1 edition. Jossey-Bass, San Francisco, CA

6. Learning Principles-Teaching Excellence & Educational Innovation - Carnegie Mellon University. http://www.cmu.edu/teaching/principles/learning.html. Accessed 19 May 2017

7. Alamäki A (2002) eLearning : osaamisen kehittämisen digitaaliset keinot: strategia, sisällöntuotanto, teknologia ja käyttöönotto. Edita

8. Kim W, Jeong O-R (2008) On Properly Using Technologies to Make E-Learning Effective. In: Adv. Web Based Learn. - ICWL 2008. Springer, Berlin, Heidelberg, pp 1–7

9. Schunk DH, Pintrich PR, Meece J (2014) Motivation | Education.com.

https://www.education.com/reference/article/motivation/. Accessed 1 Nov 2017 10. Interesting | Define Interesting at Dictionary.com.

http://www.dictionary.com/browse/interesting. Accessed 1 Nov 2017

11. Siddaway A How to Do a Systematic Literature Review and Meta-Analysis | Meta Analysis | Systematic Review. In: Univ. Stirling.

http://www.stir.ac.uk/media/schools/management/documents/centregradresearch/Ho w%20to%20do%20a%20systematic%20literature%20review%20and%20meta-analysis.pdf. Accessed 2 Nov 2017

12. Wei H-C, Peng H, Chou C (2015) Can more interactivity improve learning

achievement in an online course? Effects of college students’ perception and actual use of a course-management system on their learning achievement. Comput Educ 83:10–21

13. Perumalla C, Mak J, Kee N, Matthews S (2011) Integrating web applications to provide an effective distance online learning environment for students. Procedia Comput Sci 3:770–784

14. Hu M, Li H, Deng W, Guan H (2016) Student Engagement: One of the Necessary Conditions for Online Learning. IEEE, pp 122–126

15. Driscoll A, Jicha K, Hunt AN, Tichavsky L, Thompson G (2012) Can online courses deliver in-class results? A comparison of student performance and satisfaction in an online versus a face-to-face introductory sociology course. Teach Sociol 40:312–331 16. Lee J-W (2010) Online support service quality, online learning acceptance, and

student satisfaction. Internet High Educ 13:277–283

17. Chapman SA, Goodman S, Jawitz J, Deacon A (2016) A strategy for monitoring and evaluating massive open online courses. Eval Program Plann 57:55–63

18. Selvi K (2010) Motivating factors in online courses. Procedia - Soc Behav Sci 2:819–824

19. Caron RM (2013) Teaching epidemiology in the digital age: considerations for academicians and their students. Ann Epidemiol 23:576–579

20. van Rooij SW, Zirkle K (2016) Balancing pedagogy, student readiness and

accessibility: A case study in collaborative online course development. Internet High Educ 28:1–7

21. Kilicay-Ergin N, Laplante PA (2013) An Online Graduate Requirements Engineering Course. IEEE Trans Educ 56:208–216

22. Baehr C (2012) Incorporating user appropriation, media richness, and collaborative knowledge sharing into blended e-learning training tutorial. IEEE Trans Prof Commun 55:175–184

23. Yang D (2013) Instructional strategies for teaching science online. In: Front. Educ.

Conf. 2013 IEEE. IEEE, pp 1477–1479

24. Kist AA, Brodie L (2012) Quality of service, quality of experience and online learning. In: Front. Educ. Conf. FIE 2012. IEEE, pp 1–6

25. Corke P, Greener E, Philip R (2016) An Innovative Educational Change: Massive Open Online Courses in Robotics and Robotic Vision. IEEE Robot Autom Mag 23:81–89

26. de S Sirisuriya SCM, Ranathunga L, Karunanayaka SP, Abdullah NA (2013) Calibration checklist to use in an automated pedagogical evaluation process of web based E-Learning. In: Ind. Inf. Syst. ICIIS 2013 8th IEEE Int. Conf. On. IEEE, pp 162–167

27. Gamage D, Perera I, Fernando S (2015) A Framework to analyze effectiveness of eLearning in MOOC: Learners perspective. In: Ubi-Media Comput. UMEDIA 2015 8th Int. Conf. On. IEEE, pp 236–241

28. Kampov-Polevoi J (2010) Framework for analysis of online course design. Proc Assoc Inf Sci Technol 47:1–2

29. Kiridena SB, Samaranayake P, Hastie DB (2014) Instructional design for online course delivery in engineering management: synthesizing learning styles,

pedagogical perspectives and contingency factors. In: Ind. Eng. Eng. Manag. IEEM 2014 IEEE Int. Conf. On. IEEE, pp 1198–1203

30. Schnetter VA, Lacy D, Jones MM, Bakrim K, Allen PE, O’Neal C (2014) Course development for web-based nursing education programs. Nurse Educ Pract 14:635–

640

31. Rubin B, Fernandes R, Avgerinou MD (2013) The effects of technology on the Community of Inquiry and satisfaction with online courses. Internet High Educ 17:48–57

32. Chen Y-C (2014) An empirical examination of factors affecting college students’

proactive stickiness with a web-based English learning environment. Comput Hum Behav 31:159–171

33. Lin H-F (2010) An application of fuzzy AHP for evaluating course website quality.

Comput Educ 54:877–888

34. Cho M-H, Kim Y, Choi D (2017) The effect of self-regulated learning on college students’ perceptions of community of inquiry and affective outcomes in online learning. Internet High Educ 34:10–17

35. Kizilcec RF, Pérez-Sanagustín M, Maldonado JJ (2016) Recommending Self-Regulated Learning Strategies Does Not Improve Performance in a MOOC. ACM Press, pp 101–104

36. Anderson A, Huttenlocher D, Kleinberg J, Leskovec J (2014) Engaging with massive online courses. In: Proc. 23rd Int. Conf. World Wide Web. ACM, pp 687–

698

37. Bae E, Prasad PWC, Alsadoon A, Bajaj K (2015) Framework to improve delivery methods in higher education through online learning. In: Eng. Educ. ICEED 2015 IEEE 7th Int. Conf. On. IEEE, pp 130–134

38. Kessler GC (2007) Online education in computer and digital forensics: A case study.

In: Syst. Sci. 2007 HICSS 2007 40th Annu. Hawaii Int. Conf. On. IEEE, p 264a–

264a

39. Morin D, Thomas JD, Kira D (2015) Perceived impact of activities and resources on higher order learning skills in an online course. In: Inf. Technol. Comput. Appl.

Congr. WCITCA 2015 World Congr. On. IEEE, pp 1–6

40. May D, Moore SL (2012) “Engineering the future: A global Endeavor” An online course for transnational learning experiences with German and US students. Inst.

Electr. Electron. Eng. Ed 2012 2012 15th Int. Conf. Interact. Collab. Learn. ICL 2012 Villach Austria 26 - 28 Sept.

41. Jaggars SS, Xu D (2016) How do online course design features influence student performance? Comput Educ 95:270–284

42. Ambrose E (2010) Services science curriculum: Design and web-based delivery. In:

Transform. Eng. Educ. Creat. Interdiscip. Ski. Complex Glob. Environ. 2010 IEEE.

IEEE, pp 1–17

43. Duan J, Qi Y (2014) An Online curriculum design framework based on 3P model from the perspective of knowledge transfer. In: Comput. Sci. Educ. ICCSE 2014 9th Int. Conf. On. IEEE, pp 710–713

44. Lee JH (2010) A Traditional Teaching Model Embedded in Online Course Design.

IEEE, pp 493–498

45. Hackbarth AJ, Derry SJ, Eagan BR, Gressick J (2010) Adapting workflow

technology to design-based research: development of a method for organizing the messiness of research in technology-rich online learning environments. In: Proc. 9th Int. Conf. Learn. Sci.-Vol. 1. International Society of the Learning Sciences, pp 667–674

46. Daud NA, Sahari@Ashaari N, Muda Z (2013) An Initial Model of Persuasive Design in Web based Learning Environment. Procedia Technol 11:895–902 47. Sun J (2016) Multi-dimensional alignment between online instruction and course

technology: A learner-centered perspective. Comput Educ 101:102–114

48. Levy Y (2008) An empirical development of critical value factors (CVF) of online learning activities: An application of activity theory and cognitive value theory.

Comput Educ 51:1664–1675

49. McGready J, Brookmeyer R (2013) Evaluation of student outcomes in online vs.

campus biostatistics education in a graduate school of public health. Prev Med 56:142–144

50. Lu M-TP, Cavazos Vela J (2015) Online Learning Perceptions and Effectiveness of Research Methods Courses in a Hispanic-Serving Higher Education Institute. J Hisp High Educ 14:34–55

51. Keramidas CG (2012) Are undergraduate students ready for online learning? A comparison of online and face-to-face sections of a course. Rural Spec Educ Q 31:25

52. Driscoll A, Jicha K, Hunt AN, Tichavsky L, Thompson G (2012) Can Online Courses Deliver In-class Results?: A Comparison of Student Performance and

Satisfaction in an Online versus a Face-to-face Introductory Sociology Course.

Teach Sociol 40:312–331

53. Hadley J, Kulier R, Zamora J, et al (2010) Effectiveness of an e-learning course in evidence-based medicine for foundation (internship) training. J R Soc Med 103:288–294

54. Artino AR (2010) Online or face-to-face learning? Exploring the personal factors that predict students’ choice of instructional format. Internet High Educ 13:272–276 55. Paechter M, Maier B (2010) Online or face-to-face? Students’ experiences and

preferences in e-learning. Internet High Educ 13:292–297

56. Ou C, Goel AK, Joyner DA, Haynes DF (2016) Designing Videos with Pedagogical Strategies: Online Students’ Perceptions of Their Effectiveness. ACM Press, pp 141–144

57. Sridharan B, Hepu Deng, Corbitt B (2010) The perceptions of learners on the effectiveness of e-learning in higher education: An empirical study. IEEE, pp V1-167-V1-171

58. Chtouki Y, Harroud H, Khalidi M, Bennani S (2012) The impact of YouTube videos on the student’s learning. IEEE, pp 1–4

59. Szpunar KK, Jing HG, Schacter DL (2014) Overcoming overconfidence in learning from video-recorded lectures: Implications of interpolated testing for online

education. J Appl Res Mem Cogn 3:161–164

60. Li YW, Mai N, Tse-Kian N (2013) Using Mayer’s Design Principles in Online Learning Modules: Implementation in a Student Centered Learning Environment.

In: 2013 Int. Conf. Inform. Creat. Multimed. pp 304–309

61. Lehtola WI, Gemignani SM, Sutherland JT, Jeon M (2014) “Not All Visual Media Are Helpful”: An Optimal Instructional Medium for Effective Online Learning. Proc Hum Factors Ergon Soc Annu Meet 58:1351–1355

62. Manna R (2012) Effective learning through e-learning system: DEA Approach: HCI technologies for education. IEEE, pp 1–6

63. Ando M, Ueno M (2008) Cognitive Load Reduction on Multimedia E-Learning Materials. IEEE, pp 268–272

64. Kelly M, Lyng C, McGrath M, Cannon G (2009) A multi-method study to determine the effectiveness of, and student attitudes to, online instructional videos for teaching clinical nursing skills. Nurse Educ Today 29:292–300

65. Thomas RA, West RE, Borup J (2017) An analysis of instructor social presence in online text and asynchronous video feedback comments. Internet High Educ 33:61–

66. Bani-Salameh H, Fakher SA (2015) E-Learning Critical Success Factors Model:

Empirical Investigation. ACM Press, pp 1–6

67. Hasegawa D, Ugurlu Y, Sakuta H (2013) A case study to investigate different types of intrinsic motivation in using an e-learning system. IEEE, pp 362–366

68. Chen K-C, Jang S-J (2010) Motivation in online learning: Testing a model of self-determination theory. Comput Hum Behav 26:741–752

69. Shapiro HB, Lee CH, Wyman Roth NE, Li K, Çetinkaya-Rundel M, Canelas DA (2017) Understanding the massive open online course (MOOC) student experience:

An examination of attitudes, motivations, and barriers. Comput Educ 110:35–50 70. Artino AR, Stephens JM (2009) Academic motivation and self-regulation: A

comparative analysis of undergraduate and graduate students learning online.

Internet High Educ 12:146–151

71. Michinov N, Brunot S, Le Bohec O, Juhel J, Delaval M (2011) Procrastination, participation, and performance in online learning environments. Comput Educ 56:243–252

72. Yajie Wang, Hongyu Du, Ying Hao (2010) A case study of Problem-Based Learning instruction design under web-based environment. IEEE, pp V1-586-V1-591

73. Ali Z, Samaka M (2013) ePBL: Design and implementation of a Problem-based Learning environment. IEEE, pp 1209–1216

74. Mansor MSA, Ismail A (2012) Learning Styles and Perception of Engineering Students Towards Online Learning. Procedia - Soc Behav Sci 69:669–674 75. Amriani A, Aji AF, Utomo AY, Junus KM (2013) An empirical study of

gamification impact on e-Learning environment. IEEE, pp 265–269

76. Hrastinski S (2009) A theory of online learning as online participation. Comput Educ 52:78–82

77. Soffer T, Kahan T, Livne E (2017) E-assessment of online academic courses via students’ activities and perceptions. Stud Educ Eval 54:83–93

78. Huang I (2009) The effects of personality factors on participation in online learning.

ACM Press, p 150

79. Keller H, Karau SJ (2013) The importance of personality in students’ perceptions of the online learning experience. Comput Hum Behav 29:2494–2500

80. Wang C, Qi B (2009) E-learning Methods and Hindering Factors for their Usage:

An Empirical Study in an Education Institute. IEEE, pp 1060–1063

81. Yusof ANM, Ahmad NL (2012) An Investigation on the Relationship between Online Distance Learning with Learning Usability. Procedia - Soc Behav Sci 65:1066–1070

82. Yusof ANM, Kassim E, Norol Hamiza Zamzuri (2010) Online distance learning:

Quality characteristics and usability evaluation. IEEE, pp 575–579

83. Marinakou E (2013) An Investigation of Factors that Contribute to Student Satisfaction from Online Courses: The Example of an Online Accounting Course.

IEEE, pp 462–468

84. Ma J, Han X, Yang J, Cheng J (2015) Examining the necessary condition for

engagement in an online learning environment based on learning analytics approach:

The role of the instructor. Internet High Educ 24:26–34

85. Lee SJ, Srinivasan S, Trail T, Lewis D, Lopez S (2011) Examining the relationship among student perception of support, course satisfaction, and learning outcomes in online learning. Internet High Educ 14:158–163

86. Hew KF, Cheung WS (2014) Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educ Res Rev 12:45–58

87. Overbaugh RC, Nickel CE (2011) A comparison of student satisfaction and value of academic community between blended and online sections of a university-level educational foundations course. Internet High Educ 14:164–174

88. Cho M-H, Kim BJ (2013) Students’ self-regulation for interaction with others in online learning environments. Internet High Educ 17:69–75

89. Cho M-H, Kim Y, Choi D (2017) The effect of self-regulated learning on college students’ perceptions of community of inquiry and affective outcomes in online learning. Internet High Educ 34:10–17

90. Raymond A, Jacob E, Jacob D, Lyons J (2016) Peer learning a pedagogical

approach to enhance online learning: A qualitative exploration. Nurse Educ Today 44:165–169

91. Markova T, Glazkova I, Zaborova E (2017) Quality Issues of Online Distance Learning. Procedia - Soc Behav Sci 237:685–691

92. Richardson JC, Maeda Y, Lv J, Caskurlu S (2017) Social presence in relation to students’ satisfaction and learning in the online environment: A meta-analysis.

Comput Hum Behav 71:402–417

93. Campbell M, Gibson W, Hall A, Richards D, Callery P (2008) Online vs. face-to-face discussion in a web-based research methods course for postgraduate nursing students: A quasi-experimental study. Int J Nurs Stud 45:750–759

94. Arbaugh JB, Benbunan-Fich R (2007) The importance of participant interaction in online environments. Decis Support Syst 43:853–865

95. Dolan E, Hancock E, Wareing A (2015) An evaluation of online learning to teach practical competencies in undergraduate health science students. Internet High Educ 24:21–25

96. Sansone C, Smith JL, Thoman DB, MacNamara A (2012) Regulating interest when learning online: Potential motivation and performance trade-offs. Internet High Educ 15:141–149

97. Abd-Hamid NH, Walkner L (2017) Evidence-based best practices in designing and developing quality eLearning for the public health and health care workforce.

Pedagogy Health Promot 3:35S–39S

98. Danaher M (2014) Online engineering courses: Benchmarking quality. In: 2014 Int.

Conf. Interact. Collab. Learn. ICL. pp 1079–1086 99. (2017) Lecture | Teaching Commons.

https://teachingcommons.stanford.edu/gallery/lecture. Accessed 17 Oct 2017 100. eLearning. Osaamisen kehittämisen digitaaliset keinot: strategia, sisällöntuotanto,

teknologia ja käyttöönotto.

101. Mendenhall DR (2012) What Is Competency-Based Education? Huffington Post 102. (2017) Characteristics of Effective Online Assignments | The Sheridan Center for

Teaching and Learning. https://www.brown.edu/about/administration/sheridan-center/teaching-learning/course-design/learning-technology/online-assignments.

Accessed 17 Oct 2017

103. Nielsen J (2012) Usability 101: Introduction to Usability.

https://www.nngroup.com/articles/usability-101-introduction-to-usability/. Accessed 17 Oct 2017

104. Opitz B, Ferdinand NK, Mecklinger A (2011) Timing Matters: The Impact of Immediate and Delayed Feedback on Artificial Language Learning. Front Hum Neurosci. doi: 10.3389/fnhum.2011.00008

105. (2017) Motivating Students. https://teach.com/what/teachers-change-lives/teachers-motivate/. Accessed 17 Oct 2017

106. Esri (2017) ArcGIS. http://www.esri.com/arcgis/about-arcgis. Accessed 30 Mar 2017

107. QGIS (2017) Tervetuloa QGIS-ohjelmiston pariin! http://www.qgis.org/fi/site/.

Accessed 30 Mar 2017

108. Oskari (2017) Oskari Map Application Platform. http://oskari.org/oskari. Accessed 30 Mar 2017

109. Kennedy MD (2013) Introducing Geographic Information Systems with ArcGIS: A Workbook Approach to Learning GIS. John Wiley & Sons

110. Environmental Engineering Applications of Geographic Information Systems. In:

MIT OpenCourseWare. https://ocw.mit.edu/courses/civil-and-environmental- engineering/1-963-environmental-engineering-applications-of-geographic-information-systems-fall-2004/. Accessed 26 May 2017

111. A Workshop on Geographic Information Systems. In: MIT OpenCourseWare.

https://ocw.mit.edu/courses/urban-studies-and-planning/11-520-a-workshop-on-geographic-information-systems-fall-2005/. Accessed 26 May 2017

112. Geographic Information System (GIS) Tutorial. In: MIT OpenCourseWare.

https://ocw.mit.edu/resources/res-str-001-geographic-information-system-gis-tutorial-january-iap-2016/. Accessed 26 May 2017

113. Geographic Information Systems (GIS). In: Coursera.

https://www.coursera.org/specializations/gis. Accessed 26 May 2017

114. Extension-ISMT E-150: Introduction to GIS | Center for Geographic Analysis, Harvard University. http://www.gis.harvard.edu/training/credit-courses/extension-ismt-e-150-introduction-gis. Accessed 26 May 2017

115. Johdanto QGIS:n käyttöön – maaliskuu 2016 | Gispo.

116. HGO4940 - Geographic Information Systems - University of Oslo.

http://www.uio.no/studier/emner/sv/iss/HGO4940/index.html. Accessed 19 Oct 2017

117. SGO1910 - Geographical Information Systems (GIS) - University of Oslo.

http://www.uio.no/studier/emner/sv/iss/SGO1910/index.html. Accessed 19 Oct 2017 118. Screencast-O-Matic. In: Screencast-O-Matic. https://screencast-o-matic.com/home.

Accessed 19 Oct 2017

119. MP4Joiner - Home. https://www.mp4joiner.org/en/. Accessed 2 Nov 2017 120. Stack Overflow - Where Developers Learn, Share, & Build Careers.

https://stackoverflow.com/. Accessed 21 Oct 2017

121. (2016) By The Numbers: MOOCS in 2016 — Class Central. In: Cl. Cent. MOOC Rep. https://www.class-central.com/report/mooc-stats-2016/. Accessed 20 Oct 2017

APPENDIX 1. Final papers from systematic literature review on online course design and framework

Search terms Title Source

“online course” + framework

(metadata)

Adapting Workflow Technology to Design-based Research: Development of a Method for Organizing the "Messiness" of Research in Technology-rich Online Learning Environments

ACM

An application of fuzzy AHP for evaluating course website quality Science Direct