The motivation for this Master’s thesis emerges mainly from the attempt to find a way to evaluate usability in a certain context. In this case the context was to evaluate how the selected smartphone user interfaces fulfill the needs of the elderly users. The prerequisite of the evaluation was to identify the needs and desires of the target group. Based on the literature review it became evident that there is a lot of research on the information and communication technology and the elderly users. Material about the mobile phones use of the elderly is also available. However, research focusing on the smartphone usage in the context of the elderly users is limited. Part of the findings on the feature phone use can be applied to smartphone use. However, all such findings may not be directly applicable. For example, the elderly persons who belong to the active daily users group are presumably more likely using smartphones than those who use mobile phone only occasionally. The active users may also have different preferences. These differences were not emphasized in the research of Kurniawan [2008] or van Dyk et al. [2012]. Therefore, also my study relied on the general findings on the design preferences of the elderly users.
Based on the recent publications such as Inostroza et al. [2013] the need to have fast and affordable tools to evaluate usability of smartphones has grown. It has been realized that the criteria of evaluation developed for PC applications and web pages is not directly applicable to the touchscreen-based mobile devices. Smartphones are also available to a wider audience due to the low-priced models. Hence there is a demand to fulfill the needs of varied user groups. Often it is more cost-efficient to evaluate the user interface against the dedicated checklists such as the accessibility checklist by Mi et al. [2013] instead of involving the real users in the design process.
A checklist development seems to follow the same pattern in different studies: study existing material, create a checklist based on the previous findings and verify the checklist with real users [Heo, et al., 2009] [Inostroza, et al., 2012] [Ji, et al., 2006] [Mi, et al., 2013]. This thesis followed the same approach excluding the verification of the checklist with real users. The original target of the thesis was not to develop a heuristic checklist. However, the checklist was required in order to conduct the evaluation by following the usability framework and the process defined by Heo et al.
[2009]. Now that the checklist exists, it obviously needs to be validated. Validation work, however, is beyond the scope of this thesis.
A checklist for evaluating from the elderly users’ point of view could not be found in prior work. There are studies suggesting design guidelines concentrating on the elderly needs such as Al-Razgan et al. [2012] and Kurniawan [2008]. The checklists like TMD do not cover the special needs of a certain user group. For example requirements related to the haptic and auditory feedback were covered in the accessibility checklist. But the accessibility checklist does not cover some topics that were essential to the elderly users. Therefore, checklists used in this study were composed based on the information from different sources. Disadvantage of the checklists is the lack of possibility to evaluate items that are heavily dependent on the actual users. For example, text messaging was excluded from the evaluation because of this. Another challenging topic to evalute
without the involvement of the actual users is how well the user interface components like icons resemble the impressions the users have. In some cases the original division of the usability properties by Heo et al. [2009] did not correspond the specific requirements of this study. For example haptic or auditory feedback is not mentioned at all by Heo et al. [2009].
The usability framework does not restrict the methods that can be used for the evaluation. For example findings of a usability test can be classified based on the multi-level and hierarchical model of the usability framework. In practice finding a representative sample of the elderly smartphone users turned out to be challenging. As explained in the literature review both demographic characteristics as well as previous experience of the technology have impact on the empirical research. Applying the expert evaluation method in this thesis reduces the noise in the results caused by the diversity of the user group. On the other hand the idea of the expert evaluation is to combine opinions from several experts instead of making the evaluation only based on the opinion of one evaluator. Especially when evaluating the severity of the findings there can be different opinions. On the other hand the idea of this study was to compare two different user interface designs. In this case the comparison was done based on the same set of criteria and severity rating for both user interface designs.
During the evaluation it became obvious that those questions on the checklists that were unambiguous and could be simply answered yes or no, were easier to rate than those which did not have a clear answer. For example the item “Can the size of the icons be changed?” can be easily evaluated without input from the real users, whereas questions like “Is the terminology clear?” is something that only the real users can give the final answer to. The severity of the findings depends also on the case being evaluated e.g. additional information in the applications, like the time zone selection in the Android’s calendar application, may distract user or then not. This is a good example of the situation where usability issue can be identified based on the heuristic evaluation, but the severity is hard to rate without input from the real users. The rating depends also on the standpoint: is the evaluation done by considering the novice users or the experienced users. In this study the approach was closer to the novice users’ viewpoint. Based on the experiences of this evaluation it can be said that in order to make the checklists suitable for the expert evaluation they require some fine-tuning, such as focusing on the clear yes/no type of questions. In addition, the multi-level hierarchical framework appears to be rather laborious procedure for evaluating usability, if there is no a specific need to make high level comparison between the devices or design solutions.
The checklist based expert evaluation aims to reveal design flaws that impair the usability and the final outcome of the evaluation is typically a proposal how to fix the flaws. Hence the biggest benefit of the evaluation is gained during the development phase when proposed changes to the user interface can be done. In this study the emphasis was to identify favorable and unfavorable design solutions from the existing user interface designs and offer examples of those classified based on the usability indicators. The usability evaluation framework by Heo et al. [2009] opens up a possibility to compare products or design solutions in high level. However, the high abstraction
level may make the presentation of the actual findings abstract and difficult to understand. Hence, the overall usability score and the scores of the indicators alone do not lead to concrete actions without knowledge of what kind of criteria and checklists have been used in the evaluation. The usability framework offers tools to quantify and visualize usability which is especially useful for the comparison purpose.
This study may not reflect the actual opinions of the elderly users. In order to improve the validity of the research, the items inspected based on the checklists should be evaluated with real users. However, the diversity of the user group makes it challenging to have a generic checklist specifically aimed for the elderly persons. A more appropriate approach could be to have separate checklists for novice users or users with the diminished sense of sight, for example. The age as such is not the determining factor. Although, diminished sense of sight is a typical symptom of aging, it can occur in younger users as well. Likewise, novice users can be found in all age groups. An interesting topic to investigate in the future would be how to make the smartphones easy to personalize to better fulfill the needs of different kinds of users as well as how to make the personalization attractive without making the users feel that they have somehow abnormal requirements.