PORPOISE—Evaluation

In order to evaluate the system beyond lab bench testing, a proper test environment was needed. However, the large range of environmental conditions required in order to trigger the various aspects of the decision mechanism posed a problem:

How could these conditions be duplicated in a real-world scenario? Thankfully, research has already been conducted on this very issue. Kjeldskov et al. (2004) demonstrated that testing a context-aware mobile system under controlled settings (in comparison to real-world settings) does not decrease the ability of the evaluation to detect usability issues. Therefore, a suitable mock environment was set up so that the designers could demonstrate the functionality of the system by controlling the sensory inputs without modifying the results.

A total of 16 participants were involved in the study. Each of the participants was a manager who oversaw more than 500 healthcare cleaning professionals. The usage of the system was demonstrated to each participant, and the participants were then given the opportunity to test the device.

In order to ascertain the real-world viability and usability of the device, several questionnaires were reviewed. Ultimately, the System Usability Scale (SUS) (Brooke, 1996) was chosen as the most effective means of testing the system. The questions from the SUS were modified to better represent the PORPOISE system.

Additionally, two questions (numbers 11 and 12) were added to directly assess the device’s training potential. The questions and the averages of the participants’

responses are shown inTable 9.

Table 9. PORPOISE Questionnaire Results (Paper V)

Question Average Value

(0-4) 1. I think my staff and I would like to use this system frequently 3.3

2. I found the system unnecessarily complex 0.9

3. I thought the system was easy to use 3.4

4. I think that my staff and I would need the support of a technical person to be able to use this system once it has been set up for my

site 2.2

5. I found the various functions of this system were well integrated 3.4 6. I thought there was too much inconsistency in this system 0.8 7. I would imagine that most of my staff would learn to use this

system very quickly 3.4

8. I found the system very cumbersome to use 0.7

9. I felt very confident using the system, and that it provided useful

information 3.6

10. My staff and I will need to learn a lot of things before we could

get going with this system 1.5

11. My staff would benefit from using this system 3.6 12. This would serve as a good training/reminder tool 3.7

Each of the questions used a standard Likert scale ranging from 0 to 4, where 0 meant strong disagreement and 4 meant strong agreement. The questionnaire was administered after the participants had an opportunity to use and see the PORPOISE system being evaluated, but before any form of discussion.

After the questionnaire was administered, an informal discussion was conducted to gauge the participants’ overall feelings concerning the PORPOISE device. Factoring in the mean score values for each of the first 10 questions, the SUS scoring yielded a value of 90.6. According to Bangor, Kortum, and Miller (2009), this score equates to an adjective rating of “Best Imaginable”.

The last two questions, which were not part of the original SUS questionnaire, asked the participants to rate the system’s overall ability to achieve the intended outcome: that is, training. Thus, these questions directly measured the system’s potential to train workers to aid in the prevention of the spread of potentially deadly pathogens without the need for medical field testing. The mean responses were 3.6 (out of 4) for question 11 and 3.7 (out of 4) for question 12. These answers, together with the SUS results and the post-discussion, indicate an overall very positive outcome and well received prototype. The general post-questionnaire dis-cussion was also very positive, and the participating managers demonstrated perceivable excitement about using the device within their various sites and locations.

Paper V represented the final step of the initial application of the created framework. The next and final section of this manuscript will address the future direction of this research and further considerations.

7 FUTURE WORK AND CONSIDERATIONS

This dissertation has described the research path that culminated in the creation of and answers to the four key research questions presented in Table 1. This path began with an attempt to discover how to provide adaptivity in a mobile setting based on a learner’s particular learning style and contextual information. This question was successfully answered in chapter three through the development, creation and evaluation of a context-aware adapted learning system on an iOS device.

The limitations uncovered upon answering the first research question led to the second research question: How can a framework be designed and evaluated to identify current trends in context-aware learning systems? This question was answered as shown in the fourth chapter through the creation and application of a framework to identify current trends in the literature on context-aware learning systems from 2009 to 2015 (inclusive).

The next step along the research journey revealed the need for a framework to provide a repeatable means for creating context-aware learning systems. This led to the formulation of the third research question on how to design, implement and evaluate a generic framework for context-aware learning systems. The research question was successfully answered by developing said framework and applying it to the systems described in chapters five and six.

All of the above systems developed in this manuscript were based on standalone devices. Although the results of the literature review (Paper II) demonstrated that server-based systems were more common (Figure 14), the reasoning behind any architectural choice depends on when the choice was made and, thus, the hardware available at the time.

Figure 14. System Infrastructure (Number of Papers)

In the past, server-based systems used servers as a means for data storage and processing. However, today’s modern hardware now possesses both the storage

and the computational abilities to complete almost any task. In fact, today’s higher-end mobile devices possess hardware capabilities similar to those of many servers just a decade ago.

The ever-changing hardware environment was the foundation for the final research question posed in this manuscript, which, in turn, drove the final question asked in the presented research path: What does the future hold for context-aware learning systems, and what are the possible issues?

This query raises the following question: Are server-based requirements becoming obsolete as technology advances? Server-based systems ease of collaboration and device communication; yet, how can they be achieved in a day and age when communicating devices may be on different continents?

One of the possible ways of answering (and, thus, solving) the collaboration question is to employ cloud computing. Cloud computing provides both practically unlimited offsite storage and incredible computational processing power. Yet, before this technology can be adopted, several other issues must be addressed.

Paper VI attempts to outline the main issues of which one should be aware before using a cloud computing service for education. These issues range from relatively benign problems, such issues relating to bandwidth connectivity issues between mobile devices (Dinh, Lee, Niyato, & Wang, 2013), to more sinister

concerns regarding security and privacy (Hashizume, Rosado, Fernández-Medina, & Fernandez, 2013). As with any technology, the safeguarding of sensitive data is of key importance to the educational domain (González-Martínez, Bote-Lorenzo, Gómez-Sánchez, & Cano-Parra, 2015). Therefore, safeguarding and protection must be addressed in terms of both the management of cloud computing and the security of any software using a cloud service provider (Almorsy, Grundy,

& Müller, 2016). Security and access to data residing within cloud-computing are critical to educational contexts because communicated data may include student records, student accounts or student learning data. Therefore, understanding secu-rity policy concerns and potential gaps in current laws and regulations is vital to the educational domain (Jaeger, Lin, & Grimes, 2008).

However, security is not the only potential setback to cloud computing in education. Another possible setback is infrastructure limitations. More specifically, collaboration and interactivity may be negatively affected by network performance and latency (González-Martínez et al., 2015). Since cloud computing is closely related to the old client/server models of years past, it may be an interesting way to advance the field of context-aware learning systems using proven ideas.

The research and findings presented in the previous chapters and associated papers are not limited to the confines of this manuscript; they can also be used with confidence by other researchers in the field of computing. The frameworks have been demonstrated to be functional and applicable to the field and to potentially save researchers both time and effort.

The benefits of this research go beyond context-aware learning systems. The overall findings presented in this manuscript may also be of value to other research areas within the field of computing science. Possible research areas in which the research presented in this manuscript may be of value include augmented and virtual reality (AR and VR). The ability of AR and VR systems to directly interact and acknowledge users’ context information and provide relevant information could be of great benefit to further research. Additionally, as was alluded to with the PORPOISE system, the field of medical training and monitors could also benefit from real-time information based on context. However, in all cases, the potential benefits of the proposed frameworks require further validation and represent a possible direction of future study.

Although this research was limited and bounded by current hardware limitations, technologies are ever-improving. In the years to come, we may see context-aware learning technologies become ubiquitous in our daily lives. Whether in formal or informal education settings, learning should always remain a key part of our growth as a society. Integrating learning into our daily lives will allow us to continually expand our horizons and investigate the world around us.

Regardless of which path future iterations of context-aware learning systems take, the future of this field is exciting. It is the hope of the author of this manuscript that the work presented herein may nudge the field further down the research path, moving the current body of knowledge forward in a direction that helps learners advance and, ultimately, helps shape the next generation.

8 BIBLIOGRAPHY

Akter, S., & Ray, P. (2010). mHealth-an ultimate platform to serve the unserved. Yearb Med Inform, 2010, 94-100.

Almorsy, M., Grundy, J., & Müller, I. (2016). An analysis of the cloud computing security problem. arXiv preprint arXiv:1609.01107.

Alrasheedi, M., & Capretz, L. F. (2013). A meta-analysis of critical success factors affecting mobile learning. Paper presented at the Teaching, Assessment and Learning for Engi-neering (TALE), 2013 IEEE International Conference on.

Baldauf, M., Dustdar, S., & Rosenberg, F. (2007). A survey on context-aware systems. Interna-tional Journal of Ad Hoc and Ubiquitous Computing, 2(4), 263-277.

Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean:

Adding an adjective rating scale. Journal of usability studies, 4(3), 114-123.

Broens, T., Van Halteren, A., Van Sinderen, M., & Wac, K. (2007). Towards an application framework for context-aware m-health applications. International Journal of Internet Protocol Technology, 2(2), 109-116.

Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7.

Capel-Cuevas, S. C., M.P., Pegalajar, M. C., de Orbe-Paya, I., & Capitan-Vallvey, L. F. (2012).

An Expert System for Full pH Range Prediction Using a Disposable Optical Sensor Array. Sensors Journal, 12(5), 1197-1206.

Chang, S.-H., Chiang, R.-D., Wu, S.-J., & Chang, W.-T. (2016). A Context-Aware, Interactive M-Health System for Diabetics. IT Professional, 18(3), 14-22.

Chen, Z. (1994, 27 Jun-2 Jul 1994). Attaching a connectionist learning assistant to inference engine. Paper presented at the Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on.

Chu, H.-C., Hwang, G.-J., Tsai, C.-C., & Tseng, J. C. R. (2010). A two-tier test approach to developing location-aware mobile learning systems for natural science courses. Com-puters & Education, 55(4), 1618-1627. doi:

http://dx.doi.org/10.1016/j.compedu.2010.07.004

Cooper, D. (1994). A dual-sensor rate-responsive pacemaker algorithm incorporating a fuzzy logic expert system. Paper presented at the Computers in Cardiology 1994

Cope, B., & Kalantzis, M. (2008). Ubiquitous learning: An agenda for educational transfor-mation. Proceedings of the 6th Networked Learning.

Deluca, J. M., & Enmark, R. (2000). E-health: the changing model of healthcare. Frontiers of health services management, 17(1), 3.

Dinh, H. T., Lee, C., Niyato, D., & Wang, P. (2013). A survey of mobile cloud computing:

architecture, applications, and approaches. Wireless Communications and Mobile Computing, 13(18), 1587-1611.

El-Bishouty, M. M., Ogata, H., Rahman, S., & Yano, Y. (2010). Social knowledge awareness map for computer supported ubiquitous learning environment. Journal of Education-al Technology & Society, 13(4), 27-37.

Felder, R. M., & Soloman, B. A. . (1997). Index of Learning Styles questionnaire. Retrieved March 22, 2017, from https://www.webtools.ncsu.edu/learningstyles/

Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering educa-tion. Engineering education, 78(7), 674-681.

Franklin, J. E., Carmody, C. L., Keller, K., Levitt, T. S., & Buteau, B. L. (1988). Expert system technology for the military: selected samples. Paper presented at the Proceedings of the IEEE

González-Martínez, J. A., Bote-Lorenzo, M. L., Gómez-Sánchez, E., & Cano-Parra, R. (2015).

Cloud computing and education: A state-of-the-art survey. Computers & Education, 80, 132-151.

Hashizume, K., Rosado, D. G., Fernández-Medina, E., & Fernandez, E. B. (2013). An analysis of security issues for cloud computing. Journal of Internet Services and Applications, 4(1), 5.

Hong, J.-y., Suh, E.-h., & Kim, S.-J. (2009). Context-aware systems: A literature review and classification. Expert Systems with Applications, 36(4), 8509-8522. doi:

http://dx.doi.org/10.1016/j.eswa.2008.10.071

Huang, A. F. M., Yang, S. J. H., & Liaw, S.-S. (2012). A study of user's acceptance on situa-tional mashups in situasitua-tional language teaching. British Journal of Educasitua-tional Tech-nology, 43(1), 52-61. doi: 10.1111/j.1467-8535.2010.01152.x

Huang, Y. M., & Chiu, P. S. (2015). The effectiveness of a meaningful learning‐based evalua‐

tion model for context‐aware mobile learning. British Journal of Educational Technol‐

ogy, 46(2), 437-447.

Hwang, G.-J. (2014). Definition, framework and research issues of smart learning environ-ments - a context-aware ubiquitous learning perspective. Smart Learning Environ-ments, 1(1), 4. doi: 10.1186/s40561-014-0004-5

Hwang, G.-J., Yang, T.-C., Tsai, C.-C., & Yang, S. J. H. (2009). A context-aware ubiquitous learning environment for conducting complex science experiments. Computers & Ed-ucation, 53(2), 402-413. doi: http://dx.doi.org/10.1016/j.compedu.2009.02.016

Immanuel, V. A., & Raj, P. (2015). Enabling context-awareness: A service oriented architec-ture implementation for a hospital use case. Paper presented at the 2015 International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT).

Jaeger, P. T., Lin, J., & Grimes, J. M. (2008). Cloud computing and information policy: Com-puting in a policy cloud? Journal of Information Technology & Politics, 5(3), 269-283.

Kjeldskov, J., Skov, M. B., Als, B. S., & Høegh, R. T. (2004). Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. In S. Brewster & M. Dunlop (Eds.), Mobile Human-Computer Interaction - Mo-bileHCI 2004: 6th International Symposium, MoMo-bileHCI, Glasgow, UK, September 13 - 16, 2004. Proceedings (pp. 61-73). Berlin, Heidelberg: Springer Berlin Heidelberg.

Kranz, M., Möller, A., Hammerla, N., Diewald, S., Plötz, T., Olivier, P., & Roalter, L. (2013).

The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices. Pervasive and Mobile Computing, 9(2), 203-215. doi:

http://dx.doi.org/10.1016/j.pmcj.2012.06.002

Kusiak, A. (1987). Artificial intelligence and operations research in flexible manufacturing systems. INFOR: Information Systems and Operational Research, 25, 2-12.

Kusiak, A. (1989). Expert systems and optimization. IEEE Transactions on Software Engi-neering, 15(8), 1017-1020.

Lane, N. D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. T. (2010). A survey of mobile phone sensing. Communications Magazine, 48, 140-150.

Levi, K. (1989). Expert systems should be more accurate than human experts: evaluation procedures from human judgement and decision making. IEEE Transactions on Sys-tems, Man and Cybernetics, 19(3), 647-657.

Liu, G.-Z., & Hwang, G.-J. (2010). A key step to understanding paradigm shifts in e-learning:

towards context-aware ubiquitous learning. British Journal of Educational Technolo-gy, 41(2), E1-E9. doi: 10.1111/j.1467-8535.2009.00976.x

McBryan, B., Meyer, S. M., & Knabach, M. D. (1990). A sensor management expert system for multiple sensor integration (MSI). Paper presented at the National Aerospace and Electronics Conference. NAECON 1990.

Montoya, A., & Mody, L. (2011). Common infections in nursing homes: a review of current issues and challenges. Aging health, 7(6), 889-899.

Munoz-Price, L. S., Birnbach, D. J., Lubarsky, D. A., Arheart, K. L., Fajardo-Aquino, Y., Rosalsky, M., . . . Carling, P. (2012). Decreasing Operating Room Environmental Path-ogen Contamination through Improved Cleaning Practice. Infection Control &

Hospital Epidemiology, 33(9), 897-904. doi: 10.1086/667381

Pawar, P., van Beijnum, B.-J., Mei, H., & Hermens, H. (2009). Towards proactive context-aware service selection in the geographically distributed remote patient monitoring system. Paper presented at the Wireless Pervasive Computing, 2009. ISWPC 2009. 4th International Symposium on.

Perera, C., Zaslavsky, A., Christen, P., & Georgakopoulos, D. (2014). Context Aware Compu-ting for The Internet of Things: A Survey. Communications Surveys & Tutorials, IEEE, 16(1), 414-454. doi: 10.1109/SURV.2013.042313.00197

Prabhakaran, S., Sharon Rosy, S., & Shakena Grace, S. (2010). Detecting and spotting bombs using wireless sensors and expert systems. Paper presented at the IEEE International Conference on Communication Control and Computing Technologies (ICCCCT) Schilit, B., Adams, N., & Want, R. (1994). Context-Aware Computing Applications. Paper

presented at the First Workshop on Mobile Computing Systems and Applications.

Sharples, M., Taylor, J., & Vavoula, G. (2010). A theory of learning for the mobile age Me-dienbildung in neuen Kulturräumen (pp. 87-99): Springer.

Singh, S., & Karwayun, R. (2010, 12-14 April 2010). A Comparative Study of Inference En-gines. Paper presented at the Information Technology: New Generations (ITNG), 2010 Seventh International Conference on.

Stratulat, B., & Popa, M. (2011). Data collection in a hybrid architecture of mobile devices and sensors. Paper presented at the 2011 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI).

Tan, Q., Kuo, Y.-H., Jeng, Y.-L., Wu, P.-H., Huang, Y.-M., Liu, T.-C., & Chang, M. (2009).

Location-based adaptive mobile learning research framework and topics. Paper pre-sented at the Computational Science and Engineering, 2009. CSE'09. International Conference on.

Weiser, M., Gold, R., & Brown, J. S. (1999). The origins of ubiquitous computing research at PARC in the late 1980s. IBM systems journal, 38(4), 693-696.

Won, S., Golnaraghi, F., & Melek, W. W. (2009). A Fastening Tool Tracking System Using an IMU and a Position Sensor With Kalman Filters and a Fuzzy Expert System. IEEE Transactions on Industrial Electronics, 56(5), 1782-1792.

Yu, P., Wu, M. X., Yu, H., & Xiao, G. Q. (2006). The challenges for the adoption of M-health.

Paper presented at the 2006 IEEE International Conference on Service Operations and Logistics, and Informatics

Dissertations in Forestry and Natural Sciences

RICHARD A. W. TORTORELLA

FRAMEWORK FOR CONTEXT-AWARE LEARNING SYSTEMS

PUBLICATIONS OF

THE UNIVERSITY OF EASTERN FINLAND

DISSERTATIONS | RICHARD A. W. TORTORELLA | FRAMEWORK FOR CONTEXT-AWARE LEARNING SYSTEMS | N

uef.fi

PUBLICATIONS OF

THE UNIVERSITY OF EASTERN FINLAND Dissertations in Forestry and Natural Sciences

When using mobile devices for learning the context of the learner can change. This change may affect how and what is learnt. This disser-tation provides a view into the field which in-vestigates this effect: context-aware learning.

A framework was developed and used to create two prototypes. Their successful

imple-mentation and testing shows the overall effectiveness and usability of the framework as

a research tool in the development of con-text-aware learning systems.

RICHARD A. W. TORTORELLA

In document Framework for context-aware learning systems (sivua 66-76)