An Augmented reality concept for a restaurant’s menu based on existing user experience data

(1)

An Augmented reality concept for a restaurant’s menu based on existing user experience data

Sarwer Sakib

Haaga-Helia University of Applied Sciences Bachelor’s Thesis

2021

Bachelor of Business Information Technology

(2)

Author(s) Sakib Sarwer Degree

Bachelor of Business Administration (Business Information technology) Report/thesis title

An Augmented reality concept for a restaurant’s menu based on existing user experience data

Number of pages and appendix pages 45 + 7

Concept development is a product development stage that requires in-depth user research and analysis which helps to prioritize user needs and challenges. It is a development process that provides the basis in designing a robust application.

Augmented reality has been a rapidly growing technology since its emergence and slowly but gradually different industries are adapting this technology into their product and service designs. But the case is not same for restaurants as there has not been much work or research done to explore the opportunity of utilizing this superior information technology.

And in the post Covid-19 era this is something that should be taken into consideration.

This report was prepared to explore what are the existing methods and technologies that are used in the case of restaurant menu designs and what are the ways Augmented Reality can be implemented to enhance the existing user experience.

The research was done in four parts. A thorough literature review was made to understand different concept related to the research. Then a qualitative and quantitative analysis was done to understand the current scenario in restaurant industry and to explore the

frustrations, needs and preferences of users wrt. ordering channel designs. Using these analytical data a conceptual design was then made and finally the design was tested by users to explore the usability and validate different aspects of the design.

The research took about 5 months to complete and ‘the author’ with the help of his ‘thesis advisor’ was able to successfully conduct the research and reach most of its goals.

The result of the research was quite satisfactory as it provided substantial knowledge about the industry status, and user expectations of restaurant menu design and the prospects of Augmented Reality in the development of the designs for the future.

The ‘thesis author’ first of all praises and thanks the Allmighty for all the provisions. He also shares gratitude to his supervisor ‘Dr.Amir Dirin’ for all the support he provided during the research period. Appreciation also goes to all the supporters who supported and motivated him. And ‘Taco bell Finland’ for providing valuable data that helped the research to reach its goals.

Keywords

Augmented Reality, UX/UI, Restaurant menu design, UCD, Usability test

(3)

1 Introduction

Information accessibility is a key aspect in this consumption focused global economy where product/service providers are competing against each other to make their product/service as much attractive as possible. In a world where consumers are trained and equipped enough to access adequate information about a product, it led to industries where this competition is extended so that businesses are trying to provide more and more information to the users. Newer technologies are complementing this event by making it easy for both parties to access information and proper communication. One of the recent technologies which is allowing the user to access further information than existing one’s is Augmented reality (AR). (Seer & Avornicului 2015, 98.)

Augmented reality (AR) is the implementation of virtual objects on top of the real world through digital lenses. It allows user to implement virtually developed ideas in the real world. AR provides functionable links for the user that can be accessed directly and auto- matically between the physical world and electronic information. The potential for this technology is immense which is being explored and implemented by architects, designers, and researchers to enable an advanced digital lifestyle for users where high information flow happens between user and system. (Dieter & Tobias 2016, 2.)

The use of augmented reality in various industries has been increasing in recent years due to its huge acceptance from the users and a technological platform which already is established for such technology. It has emerged as a rapidly developing technology with huge potential for the retailing with the prospect of enhance user experience and selling environment. Both the buyer and seller will get their own share of benefit from this technology as it will create a wider platform of business and transaction. However, there is yet to be enough academic research or practical implementation which is holding the opportunity this technology holds. (Francesca, Gary & Lee 2018, 119.)

In retail business AR has been gaining popularity due to its very nature of allowing users to access additional information from its ancestor technologies. Industry giants like IKEA, Amazon, Nike etc. has already adopted this technology successfully which is becoming a path for others to follow in adopting this technology in their retail business (McKinnon May 2020). Just having a smartphone now enables customers to access information about the dimension, colour, fitting and a sense of closeness in terms of getting acquainted with a product which allowed users to choose the product they like the most. However, this adaptation is still very limited to a small sector of industries, while the observed prospect of

(6)

AR in other industries such as restaurant or online food delivery is encouraging to make further study in this topic.

Restaurant business has seen a dramatic change in sales channels after Covid-19. Cus- tomers are preferring nowadays to order food home rather than physically visiting a restaurant. According to Statista, Mckinsey & Morgan Stanley, In US the sales from food delivery has increased from $16.8 billion to $26.5 billion during the period 2018-2020 (Curry September 2021). With the growth of sales there has also been many service providers who emerged as third-party service provider to deliver food from restaurants to the consumers. The success and competition of this service design indicates it will be a continuous sales medium for this industry where investors can invest more to improve the facili- ties and information accessibility. On the other hand, customers who are currently visiting the restaurant has been preferring to order from mediums which has fewer contact with other individuals.

By its nature, AR in restaurant industry can provide a superior communication channel between the restaurant owner and consumer than existing ones. Consumers will be able to access information that are not available in current technology about the product and service. This will then improve the trust and willingness for the users to use online ordering as they can be assured beforehand about the product they are planning to order. So, a user-friendly mobile application which augments the food beforehand is bound to improve the user experience in food ordering. Hence, an in-depth study about a 3D menu design for the online ordering platform to explore the opportunities and to demonstrate scope for businesses is quite interesting and encouraging to take. (Timothy 2018, 134.)

Shneiderman, Plaisant, Cohen, Jacobs & Elmqvist (2018, 131-137) suggest that to design a user interface requires knowledge about multiple designing tools and methodologies.

Terms such as User centred design (UCD), User research, User experience improvement are quite dominant in designing any interface where user interact with a system. While knowledge about requirement analysis, design environment and design tools are also mandatory to design any prototype that can demonstrate the scope and opportunity of that product.

Through this research, the goal is to understand the status, obstacles, and opportunities in the current ordering mediums in restaurant by evaluating user experience & preferences

(7)

from available technologies. While the scope of an AR based 3D menu design will be ana- lysed by designing and testing formulas. This document will be a foundation for future research about AR in restaurant industry and for learners in use of academic purposes.

(8)

2 Research questions and research methods

Researching any matter requires the researcher to fulfil some pre-requisites which helps to formulate the path and specific goals for the research, which will then be used as the measuring scale to measure the outcome of the research. For this research, some research questions will help to evaluate the outcome of the thesis as it will provide some specific goals to achieve within specified boundaries. While some pre-defined method will help to formulate the path through which the goals are expected to be achieved.

The primary goals of this thesis are to evaluate the user experience with current restaurant menu that are available to use and based on the result design a 3D menu with augmented reality technology which targets to enhance the experience of the user by providing superior information transaction. And finally evaluate the new design and justify the need for improvement of restaurant menu systems.

2.1 Research questions

The questions that this thesis is aiming to answer are listed below:

• What is the user engagement levels and user experiences with current ordering channels in restaurant industry?

• What are the challenges, gaps, and opportunities for restaurant industry in ordering systems design?

• What is a good AR concept for restaurant menu design?

These are the primary questions that this thesis will be structured upon and will try to answer. Through answering these questions scopes, challenges, and preferences for service design of ordering channels will be realized and by utilizing the user engagement and user experience a better understanding of opportunities will be made which will then be used to design an AR based ordering system to provide an improved user experience for this industry.

2.2 Research methods

Research method provides a path for researchers to follow in the search of research goals and gives guidelines which helps to do step by step research and achieve the desired outcome.

To achieve the goals of this research there are multiple methods that will be followed. Ini- tially a combination of qualitative and quantitative research about current industry status

(9)

will be made to find out the status of customer engagement and the experiences with current ordering systems of restaurants. The goal is to get data about available ordering systems and user engagement using those channels. By gathering a large amount of data about the engagement of users with different ordering systems and the issues with them, a proper quantitative analysis can be made about the user preferences in terms of different ordering channels and number of issues that has been experienced.

While a qualitative method using interviews of a group of customers can be a good source of user experience evaluation. By formulating a questionnaire which is related to the research will provide substantial amount of information both in terms of understanding the status and gaps in user experience and the basis on which the improved design can be developed to enhance the experience of the users.

After the initial research about the current industry status and user experience with the restaurant menu, the research is expected to gather enough information about the user obstacles, gaps, expectations, and preferences in using a restaurant menu. This

knowledge will then be utilized to the next step of this research- conceptualizing, designing, and evaluating an AR based restaurant menu.

To design or formulate anything it is necessary to understand the concepts of related tools. In the case of this research, it is also important to have a proper understanding about different design tools and design methods. A proper literature review should help the research in finding this knowledge.

Literature review about some related concepts such as user centred design (UCD), interaction design (ID), User experience (UX), User interface (UI), Augmented reality (AR), mobile augmented reality (MAR) etc. will be reviewed and used to design a restaurant menu based on AR to provide an enhanced information flow and superior experience.

Finally, observation method will be applied to observe the performance of the developed design utilising usability tests, which will provide the necessary knowledge about the AR design and thus allow to answer the research questions that are formulated earlier.

(10)

3 Literature reviews

To gain proper understanding about various related concepts which are fundamental in achieving the goals of the thesis, some literatures were reviewed and key ideas from them were noted down. The knowledges from these reviews are used throughout the course of this thesis in identifying user engagement levels as well as designing and testing the pro- posed AR design.

3.1 User engagement and User experience evaluation

User engagement and User experience are terms used to observe and analyse the customer interaction with different service channels of a business and the level of performance of each design (Roy & Bob, Chapter 1). As it sounds the user engagement study focuses on how to keep a user engaged with a design while user experience focuses on what would be the experience of a user when he/she interacts with the design during various time of engagement. By understanding the fundamentals of these concepts and their applications the objective of evaluating current service designs for various sales channels will be fulfilled. While this knowledge will also help to gain insights about scopes and user preferences for the AR menu design.

3.1.1 User engagement

Mounia, Healther, & Elab (2015, 3) define user engagement as, the emotional, cognitive, and behavioural activities that a user goes through which brings the attachment with any design, during the use of a resource at any time. While V. Kumar (2013, 2) illustrates User engagement as a user centric approach where companies design their product or services keeping the user as the focal point and evaluates the designs based on the usage or buying amounts. So, User engagement studies helps designers design a product or service in a way that considers users of that design as the central point and strives to make users feel engaged to the system by providing value at every moment during the interaction.

Thus, fulfilling the needs and aspirations which keeps the user connected.

To analyse the status of user engagement of any system user engagement analysis is used. User engagement analytics is a way to measure the level of user engagement with different sales channels and get the ideas about the status about the level of performance of each of those channels. (Omer 2014, 1-2.) So, by analysing the performance of service/sales channels in the restaurant industry engagement of customers can be measured and the preference of customers in different channels can be observed.

(11)

3.1.2 User experience

While user engagement is a value-driven approach to evaluate performance of a product or service, user experience is the perception of user’s interaction with a product/service (Dina 10 October 2019). Law, Roto, Hassenzahl, Vermereen & Kort (2009, 720) define user experience as the measurement of usefulness, usability, and desirability of something. So, user experience measurement will provide the performance in interaction of users with products and services and will provide knowledge of design attributes.

User experience deals with how a user experience at different stage of interaction with a system. Interaction design foundation (2021) suggests that user experience is the basis of user centric design. The design process revolves around researching, personalising, wireframing and/or prototyping while keeping the users at the centre of every decision.

While at the same time this design process also involves the integration of product values, branding, emotions etc.

When designing a system, user experience should not be considered separately for different stage of interaction. User experience design is a cohesive process and deals with the integration of the experiences of the user. As Norman (December 2018) suggests, while designing something it is important to make sure that the designer considers the journey of the user from the initial stage to the final stage of the interaction. He emphasised that, the designing process should maintain cohesion to give a continuous experience to the user during every involvement with the design.

User experience can be measured by doing a proper user research, where a systemic study about the goal, need and obstacle of users would be measured (Jeff & James.

Chapter 2). As mentioned earlier it is a user centric study. So, by doing a user research about their experience with different ordering mediums will provide knowledge about the issues, needs and expectations of users from an ordering medium which will be used as basis of designing a new AR based menu.

(12)

3.2 Designing a user interface 3.2.1 Design basics

Designing a new system requires to follow some empirical evidence and theories which were accumulated by experienced designers and researchers. There are guidelines for the designers which are low level focused advice intending to act as cautionary to dangers and suggests good practices. (Shneiderman et al. 2018, 82-87.) Concepts such as how to navigate the interface, Organizing the display, how to get the user’s attention, how to facil- itate data entry acts as guidelines from different authorities who oversees the application development process. Issues such as allowing disable individual to access the product/service, consistency of data display and some cautions about over-elaboration acts as guidelines for the designers to follow.

While guidelines are specific to design an object and varies for design of another object.

There are also some design principles which are more fundamental, widely applicable and enduring compared to the guidelines. Guidelines are project/case dependent, but principles are more enduring and independent to any specific project/case. For example, deter- mining the skill levels of users is key to design any product/service, it allows designers to understand the level of difficulty that can be applied while designing. (Shneiderman 2002, 1-2.) An application for kids to learn alphabets can never be of same difficulty level as a CRM application, while a CRM application should be enough sophisticated to ensure every necessary component is available for the users to access to. Design principles allows to determine the user skills in designing an appropriate design.

Another principle to design any system is to identify the tasks that the users are supposed to execute to reach the goal of the system. Norman (2013, 1-2) shared the necessity of task definition by illustrating that famous picture of a coffeepot from French artist Carel- man which has both the handle as well as the spout on the same side which made the pot unusable. This was a great example of a design which was not focused on task identification, hence made the design useless. Shneiderman et al. (2018, 90-91) suggest that task identification should be given importance and tasks which are more frequent should be categorized separately from the ones which are less frequent. This will allow the designers highlight the controls to perform frequent tasks and make it easier than less frequent tasks.

(13)

3.2.2 Fundamental rules of design

Shneiderman (2018, 95-97) suggested eight golden rules for interaction design. These rules are generally applicable to any design while needs fine tuning for specific purposes.

The first rule is to maintain consistency in design. It is necessary to maintain consistency when designing similar actions. Everything around us in the nature is so organized that subconsciously we expect things around us to be consistent and whenever there is some inconsistency it triggers alarm to our brain and causes arousal with some uncomfortable feeling. A common example can be that every book in English we read are organized from left to right. If there is a book which is designed other way (right-left) it will not be consistent with the everyday experience of users and will lose its value. Secondly, A design should be usable at different situation and allow users with different usage capacity to access its potentials. A term known as universal usability is to understand the difference among different user groups and their requirements and design in such a way that attract users by realising their needs. Third rule is that designs should be communicative and always let the user know the status of the user by providing informative feedback. When designing a user interface of restaurant menu users must always be informed at the stage they are in, what they have accomplished and what can they do. Another rule is that a design should be structured such that users can perform a task without any interference of other activities which requires designers to design sequence of activities in groups. This is very important aspect specially designing a sales channel as the expectation of the restaurant owners is that customers buy what they want and flawlessly. For example, having an advertisement or window text during ordering will surely give user a bad experience and thus lose its value. Error prevention should be considered while designing. According to Shneiderman (2018, 95-97), Designers should make sure that users can reverse their action in case of any mistake or if the user changes his mind about the action that was performed earlier. The seventh rule is the users should be given control of the task for better user engagement. Naturally we love to be in control of our surroundings and when we feel we have control over a system we tend to feel more comfortable and engage with the system better. And lastly, designs should be such that the users are not required to re- member much and can focus on instant activity hence improving the performance of actions. These are the rules which are very general and required to be assessed for every individual and specific situation.

Additionally, Norman (2013, 4) suggested discoverability and understanding of a design by its users as the fundamental aspects of any design. This emphasis was also given by Krug (2014, Chapter 1) who describes this from the perspective of web page, that a web

(14)

page should be self-explanatory and understandable by a new user without any difficulties. Lidwell et al. (2003, 16) asks designers to design in such a way which is already known to the user and this way the accessibility of information will be higher, hence improving the discoverability and understanding of the design.

3.2.3 User centred design (UCD)

User centred designing focuses on users, their behaviours, preferences, and dislikes and allows designers to design in a way that suits the needs of the user and make them connected to the design. For example, the widespread usage of smartphones and other smart devices with various functionalities allow us to effectively complete tasks in different areas of our life has made these devices more than just some electronics, rather an extended artificial organ to our body. This behavioural change has not been existing for long, merely since last decade. Companies have realized this change and started designing their products that makes user more attached to their product. For example, Industry giant ‘Tesla’

who produces different electric machines has considered this behavioural change of hu- manity and applied them to change the design of their appliances. ‘Tesla’ has redesigned the traditional internal control system of cars by getting rid of all manual controls and bring them all under a touchscreen control panel, which is a very common device for everyone nowadays and specially someone who can afford a Tesla (Tesla, 2021). This was done only by realising the design based on user behaviour analysis under the study of user centred design.

Norman (2013, 8) defines user centred design as a design process which keeps the needs, priorities, capacities, and behaviour of users as the focal point. So, designer should always consider these necessities and attributes of users when designing anything. On the other hand, international organization for standardization (ISO 9241-

210:2019) defines this concept as the method of making any system usable by performing multi-disciplinary actions and through incorporation of factors related to knowledge of human and knowledge about ergonomics.

User centred design is a data driven approach (Lowdermilk 2013, Chapter 2), the designs are primarily based on data about the behaviour and need of target users. This helps to eliminate unnecessary components in the design and to generate a design that suits most of the target users. At the same time, this design approach is also observational (Norman 2013, 9). Users are often not aware of what they want and which kind of difficulties they are getting through. So, an observational study about user behaviour can give designers

(15)

knowledge about the perception towards the design from end users perspective, the goal is designing a minimalistic design that is easy to use but provides necessary solutions.

There are other benefits of User centred design alongside the abovementioned. Usually, a design which is not based on user centred design methodology is subjected to evaluation during the end stage of development and often brings cost and time waste due to flaws and integration issues with the design (Lowdermilk 2013, Chapter 2). But having a design that is focused on users’ issues helps in the reduction of development time and cost which mostly occurs during the early stage of software development (Shneiderman et al 2018, 137). So, by making better decision with user centred design process designers are ex- empted from designing something which is not useful.

Righi & James (2010, 127) details the activities for a user centred design approach. For an existing design evaluation and improvement project such details are valuable to use.

First phase of the study is focused on usability study of current product by making ethno- graphic interviews. Which should provide the designers knowledge about status, gaps, needs and opportunities for the new design from the perspective of users. Secondly, Task analysis need to be taken place and how to execute them should be designed using interaction design. Then, heuristic evaluation, usability test of prototypes and style guide needed to be done and finally when the functional prototype is ready with visual designs another usability test needed to be carried out to evaluate the performance of the final prototype.

3.2.4 Emotions and experience design

We humans are emotions driven creations. We experience emotional changes during different seasons. Hot and sunny day boosts energy hence people go out for picnics and holidays. While cold winter nights are rather depressional which is caused by lack of the former. Having a praise from our counterparts makes us happy and enhances the mood, while losing a game makes our emotion full of disappointment.

It was found through a series of experiments that, emotions impact our cognitive capacity which in turns controls over the effectiveness of our actions (Isen, Daubman & Nowicki, 1987). During the experiments, different group of subjects were given different emotional experience and was asked to carry out similar task. It was found that, given a negative emotional experience damages the natural ability of conducting a task, while a positive experience elevates the cognitive capacity and allows humans to perform better than their regular state.

(16)

Emotions have strong command over user engagement. Norman (2004, 3-4) describes this phenomenon by illustrating his collection of 3 teapots. Each of them has different functional obstacles which led him avoid using them for daily use and rather use a very basic tea maker for his everyday use. But due to the emotional engagement to those three not fully functional teapots he keeps them attached and puts them in display which gives him pleasant feeling.

Hassenzahl (2010, 3) views emotions as part of an experience. Depending on whether an individual experiences something which makes him feel comfortable or uncomfortable will eventually decide his emotional engagement. So, user experience design can control emotional state. This argument was supported by Daniel (28 September 2016) who illustrates current marketing techniques, designed for the consumers to build a long-term relationship with the company. So, user experience design has strong relationship with emotional design.

According to Schwartz (2017, Chapter 1) experience design is a holistic approach, is long term, and depends on relationship between a design and the user over that period. If the user gets negative emotions in any part of the interaction, it might affect their buying decisions. It is necessary to keep designing positive experiences during the whole interaction period which in turn will create positive emotions, thus make attachment with the user.

User experience and emotional design is also user centric. According to ISO 9421 an interactive system requires user contribution during the development process. This process incorporates user feedback, thoughts, feelings, and opinions regarding the system and use them to evaluate the system performance. Utilising concepts such as Nielsen’s heuristic evaluation, emotional engagement analysis etc. user emotional experience could be observed during the usage of a system and could be reflective to the design performance (Nielsen Norman group, 2020; Dirin, Laine & Nieminen, 2017).

3.2.5 Three levels of emotional design

The capacity of brain and information processing ability varies among different creations.

Norman (2004, 21) defines these processing capacities in three different levels. Visceral level- which is more subconscious and automatic level, behavioural level- a level that pro- cesses everyday behaviour and finally reflective level- which acts mostly as the analysis tool which analyses previous experiences or surrounding components and provides reflection to act upon. As mentioned, visceral level is instantaneous, it makes instant decisions about the acceptance of anything and provides immediate response to our brain and body

(17)

parts to act upon. While behavioural level is based on behaviour or practice. The brain re- ceives behavioural signal and sends them to the part which has a set of instructions about what to do during that situation. And reflective level is the highest level of processing. It is also slower in nature that the former one’s. It does not control any sensory input or behavioural activity directly but provides judgement and analysis to the behavioural level to pon- der upon and act accordingly. To fulfil the requirements of these three levels of brain processing three design concepts are used.

Visceral design addresses the need for visceral part of our brain. The nature around us is very symmetrical and has attractive elements to attract each other. A male bird or animal attracts his counterpart female animal or vice versa. Or a bee that is attracted to flower which has sweet fragrance. An animal is also attracted to a sweet fruit or tasty food. This behaviour isn’t uncommon to humans either. We like to see beautiful symmetrical things, taste sweet or delicious food, listen to beautiful voice or sound, touch smooth surfaces etc. In design psychology, visceral design deals with how to make a design attractive or likeable to the user. By using appropriate color, shape, size, and appearance a design can be made attractive to the users. Look, feel and sound dominates the visceral level of design, as it is all about initial reaction. (Norman 2003, 68-69.)

Behavioural design deals with the effectiveness of the design to the experience of the user. Norman (2003, 70-83) describes behavioural design through four components. Each of these components can impact the behavioural part of the brain and effect the emotion of the user. Namely- Functionality, understandability, usability, and physical feel. Function- ality is a key element in design. Having a design with excellent visceral design but lacking in functionality could make it unwanted and will lose the value it was intended for. Func- tionality helps to solve users’ problem in the context whether the user is aware or not. For example, a restaurant menu even if it was designed with the most aesthetic-ness as possible but doesn’t allow the user to order the food will not be of any value or use. Norman shares two way of functional development- enhancement and innovation. Both ways are reflective to users’ behaviour and solves either an existing problem or a problem which was unaware of. Understandability is the second component of behavioural design. To make a design effective it is must that users are provided with enough information to be able to understand the design. The faster a user can interpret a new design corresponds to the level of effectiveness of that design in terms of understandability. Norman advice designers to design the models such a way that there is high percentage of similarity among the models during different stages of development. By developing prototypes and

(18)

testing them understanding of a design from the user’s perspective can be understood. Af- ter that comes usability. It is possible that a product does its required tasks and understandable but might not be usable. An example of usability can be an instrument which is requires years of practice to gain control over, despite its functionality and understandability it might not be usable to many. Universal usability is very important topic in system design and is taken care of by the designers to make sure system is usable by all group of users. Different authorities also require designs to have specific usability aspects such as voice command for blind users. Norman describes Physical feel as the fourth component of behavioural design. By changing the position of an element in the design can have a different feel to the user. An example of such component can be the fingerprint sensors on the smartphones. Designers have tried to design it in many ways but eventually found the upper backside of the phone to be the best position in terms of physical feel for this option, and this is being used by almost every android designer till date. These four components when considered well during the design process, would fulfil the behavioural need of a user, and enhance the experience and engagement. Norman (2003, 70-83.)

Reflective design level deals with reflection about a product or service from user experiences, memories, preferences, and values. This design level provides message, culture and meaning to the user when interacting with a design. Norman (2003, 84-85) gives an example about two watch to illustrate this phenomenon. He details two watches one having fancy interface provides reflective delight using unconventional method of displaying the time. And the other one a practical, sensible watch without any reflective design but absolutely fulfils the requirements. Among these two watches the former despite lacking behavioural aspects in contrast to the latter is more appealing due to its reflection to the user as a mean of showing off their prestige or holding something which is not common.

While the other one despite having the utmost behavioural aspects lacks in reflective aspects. Though this argument from Norman can be counter argued, as someone who considers that second watch to be more useful and likes the brand might reflect upon those attributes during the purchase of the product and thus pondering upon the reflective design of the product. Brand images, celebrity connections, reliability, word of mouth all could be related with reflective design as these aspects do influence over the buying decision when user reflects from these aspects. Norman (2003, 88) also argues that customer relationship plays a vital role in reflective decision making, so user experience design goes hand to hand with reflective design.

(19)

3.2.6 Interaction design

Among the many aspects of user centred design process is to design how and where the user will use services to accomplish the desired task through any system. It is necessary first to evaluate the activities for which the system is to be designed and based on those activities designers should work on which medium and method would best suit the project.

(Sharp, Rogers & Preece 2019, 7.)

A popular Human centred design expert (Jamal Nichols 19 October 2017, min 01:00- 03:30) defines interaction design as the design of a reciprocal communication between two parties to achieve a specific goal. He suggests human interaction design should always be designed keeping the basic human behaviours in mind and when designing interaction, it should be made feel like a natural conversation. Sharp et al. (2019) defines Inter- action design as, to design interactive products to support the communication of people and interaction in day-to-day life. Both definitions support the concept of communication between two parties to achieve something. This is the core idea of Interaction design, to make communication channels and methods in a way which will provide a great user experience following the guidelines.

To design an Interaction system the first task is to establish the requirements. The designer must identify and establish what should the system do, so that the best possible interaction method could be fitted for the purpose and experience of the user. Establishing requirements should be based on the user characteristics, activities and environment or method of the interaction (Sharp et al. 2019, 9-10). It is also important to design alterna- tive ways to interpret and satisfy the need for an interactive product. Following design principles a designer should explore different mode of interaction, prioritize different styles and interface type for any interaction design. Sharp et al. suggests designers to sketch the designs and prototype to the best possible options to figure out the potential of designs that suits best for the user. Trying out prototypes help designers to explore different aspects of the design. And when prototypes are designed it is important to evaluate them to assess the status, gaps, and opportunities of that design. Evaluation helps realizing the extent a prototype meets the identified requirements, to identify any unidentified prospects and to establish any necessary changes to the prototype that would fit the need of the design.

Interaction design can be of different types. Sharp et al. (2019, 81-88) divides the interaction types in five different parts. Depending on the user’s action each of these types helps

(20)

designers design a way of Human-Computer interaction (HCI) to perform a specific task.

These interaction types might not be mutually exclusive or definitive.

• Instructing interaction style is when a user instructs the system by commanding through typing or speaking, choosing menu options, using gesture or button etc.

User is the initiator of these kind of interaction.’

• In conversing interaction style users have conversation with the system. Both the system and the user can be the initiator of these kind of interaction type. Normally the medium of communication is through text or voice command.

• Manipulating interaction means user manipulates any objects (physical or virtual) to change the state of the object. User is the initiator in this kind of interaction.

• Exploring interaction happens when a user interacts with a system to explore the environment virtually or physically. Technologies such as AR, VR are based on this type of interaction.

• Responding interaction occurs when the user responds to a request or query. Sys- tem is generally the initiator in these kinds of interaction and user responds to that.

3.2.7 Visual design

Visual design is to use of visual elements such as- shape, size, color, orientation, texture, position, text, typography etc. for an enhance usability, user experience and clear communication (Cooper, Reimann, Cronin & Noessel. 2014, 406-410). A visual design is a goal- oriented approach where designers present behaviour and information in a useful and understandable manner which considers aspects which are related to branding and per- sona’s goals. According to Cooper & al. (2014) aesthetic design is also considered in visual design but in a goal-oriented manner which prioritizes understandability and usability as the major subjects.

The element in visual designing has multiple properties, when different visual elements are composed together with appropriate properties the output will provide a user with meaning about the object and communication will be optimized. For example, when a user realizes about a button in an interface by its shape and color, the next time they see an object having similar shape and color right away they realize that it’s a button. By differen- tiating objects through different shapes and colors users develops greater understanding and gains better usability when interacting with a system. (Cooper & al. 2014, 406.)

Cooper & al. (2014) describes different visual elements that are used by the designers to design an effective user interface. First of them is shape of an element. Shapes are two dimensional items that helps users distinguish that item from the adjacent space. It is the primary attribute to an item. A shape can be geometric or organic. Having consistency in shape provides users with easier understanding and efficiency in interaction. Then comes size of an object, size helps to prioritize items in the interface to get attention of the user. It

(21)

is a valuable property in information hierarchy that helps designer to structure information in appropriate manner. The choice of color is also a very important decision in interface design. Depending on the user’s goal, content, brand, and environment colors should be different and designers should consider them seriously. Color also helps designers to highlight objects and get users attention. Contrast helps users identify important objects fast and efficiently. Whether an object is oriented towards down or up also defines the goal. For example, a downward arrow indicates something is going down and the opposite means its improvement. Texture is the surface quality of an object. It is used to convey dif- ferences and get user attention. At times designers use them to provide a 3D feeling to an 2D object. The next aspect of a visual element is the position of that item. The primary use of position is to convey information hierarchy in a screen. It also helps users locate element based on the position that element is normally located. For example, contact information about companies are usually located at the bottom of a web page, so a user can directly move there to find out that information. Typography creates the mood and empha- sizes the message for a user. It is known that people normally recognize words primarily by their shapes, so distinctness provides ease in reading. In interface design texts are normally used in high contrast and complementary colors are avoided due to improve readability. Appropriate typeface and size improve readability and understandability thus helps improve user experience. And it is better to user as less text as possible while main- taining the proper conveyance of message in an interface. (Cooper & al. 2014, 406-410.) User experience designer and researcher Wilbert O. Galitz has provided some design principles for visual designing which helps designers to construct a visually pleasing com- position. They are- Balance, symmetry, regularity, predictability, sequential, proportion, simplicity, and groupings. These attributes and principles help with information architecture, aesthetic-ness, understandability, usability, and user experience. (Galitz 2007, Chapter 2.)

3.3 Extended reality (XR)

Extended reality (XR) is a term to cover technologies which combines real world and virtual world together to provide a user with an extended information access. Technologies such as Augmented reality (AR), Virtual reality (VR) and/or Mixed reality (MR) all are under the umbrella of XR, that either allows user to augment anything to the user’s surroundings or immerse a user into a virtual environment. (Gerencer April 2021.)

(22)

3.3.1 AR & VR

Among the different technologies of XR, Augmented reality (AR) promises to provide interactive experience in the real world through digital lenses, so that users can access virtual information that enhances their experience in terms of information accessibility (Bansal P 2021, Chapter 1). On the contrast, virtual reality (VR) doesn’t allow a user to interact with real world but immerse a user to an environment which is synthetically created for an experience either by recreating an image of a real world or by a completely virtual design (Kipper G & Rampolla J 2012).

Both technologies have emerged with substantial promises in the retail industry to provide users with opportunity to access information beyond the capacity of other existing technologies. An example of AR technology can be illustrated is IKEA PLACE which is a mobile application designed with AR technology that allows user to augment the furniture of IKEA in their home environment through the lenses of their mobile devices (IKEA 2021).

Though due to constraint of the devices which support this technology, this is yet to be used by mass people, but users of this app have received it with positive experience which was found in the rating of this app in app-store of Apple (App store 2021). On the other hand, virtual reality requires a special device which is generally mounted on the head of the user to obstruct them from viewing the real environment and instead visualize a virtual environment that the user feel to be located in (Parisi T 2015, Chapter 2).

There is a usual association between Augmented reality and visual augmentation where there is a combination of visuals from real world and objects that are generated virtually.

But Azuma (1997, 2) argues that AR is more that overlay of computer graphics but have some characteristics which makes it superior over other related technologies.

• Combines real and virtual – It is a primary characteristic of AR that a virtual object would appear as a real-world object by occupying physical space around us.

• Interactive in real time – Another major attribute to AR is that the objects are interactive in real time. Augmentation of virtual elements in a pre-recorded video would not be considered as AR.

• Registered in 3D – It is necessary for AR devices to have 3D location tracking ability for being able to map the real space to augment virtual elements.

Registration of virtual objects in 3D space is done by different types of AR. Whether to use a marker or not to track 3D locations, an AR can be subdivided into two categories.

Marker based AR or Marker-less AR. As it literally sounds, a marker-based AR uses a marker which can be a 2D image or 3D object, the scanner recognizes the marker and augments the virtual object in relation to the position of the marker. On the other hand,

(23)

marker-less AR usually uses global-positioning system (GPS), radio-frequency identification (RFID) and/or sensor technology to control the position of virtual object. (Cheng, Chen & Chen 2017.)

3.3.2 Mobile Augmented Reality (MAR)

Anything which is mobile means that it is small to be carried with the user wherever he/she goes. So, mobile augmented reality means an augmented reality which is being operated through a mobile device. Generally, a smartphone and/or a tablet type device are considered as devices which supports MAR. (Craig 2013, 209.)

MAR has been evolving as a rapid technology since more and more smartphones are being equipped with such technology which allows users to download AR applications and use them. While the production cost is also going down which brings cheaper and afforda- ble smartphones with MAR technology for the users to buy from the market. So having an AR enabled smartphone and a stable internet connection users can experience MAR applications on the go. (Dirin & Laine 2018, 1-2.)

(24)

4 Research plan

There are multiple phases of this research, each of these phases would solve one of the research questions that were formed in chapter 2 of this report.

Firstly, to figure out the customer engagement levels with different ordering medium sales data would be collected from multiple restaurants detailing the sales through different ordering methods. This should give a good understanding about the customer engagement levels through different ordering channels and justify if an improved AR based ordering channel would be beneficial or not. Alongside, to evaluate if there is any effect in customer engagement due to covid-19 pandemic, this evaluation will consist of two different time- scale one detailing the sales of a quarter of 2020 and another showing the details with a quarter of 2021.

The research will continue to understand the user experiences with current ordering channels which will done by using a qualitative data analysis. A set of qualitative data would be gathered by making interviews with some of the manager of those restaurant who deals with customer complaints, that would provide us knowledge about what are the most common issues with customers while dealing with a restaurant. This analysis would provide us with the user experience levels of current industry channels and give emphasis to our jus- tification of designing an AR based restaurant menu design.

To solve the second question of the research, the plan is to make another qualitative analysis this time with the customers of those restaurants to gather information about the user preferences towards current ordering channels, simultaneously to explore user knowledge and feelings towards AR technology, and preference and expectations from an AR menu.

This analysis would give a profile-based knowledge about user preferences, user expectations and user needs. Which will then be the basis of concept design of the new AR based restaurant menu.

The research will continue through the conceptual design of an AR based restaurant menu using the customer analysis data which provided the knowledge about the user preference and user expectations from the new design. After preparing the conceptual design, this would then be shared with the users to get feedback which should eradicate gaps in the analysis or misunderstanding about customer expectations if any. Another qualitative re-search would be required to get this knowledge and the result would be used to design the Hi-fi prototype for AR based restaurant menu design.

(25)

The final stage of this research will be the usability test of the design prototype. This test would be useful to learn about the design performance, to understand user-experience with the design, to gain user feedback, to make necessary adaptation to user requirements and to improve the functionality of the design. A usability test would also figure out whether the design solves the related problems or not, what is the level of ease interacting with the de-sign, whether the icons are traditional or not, how consistent are the color or contrast etc. To perform the usability test with the design prototype, multiple test cases would be formed and then those test cases would be given to the users to figure out the status of the app from the perspective of the stated usability attributes. The usability tests are going to be done in a real environment and some test cases would be provided to the users to evaluate the design performance in terms of user experience and usability.

Based on the test case results the design attributes will be analyzed and pondered upon.

The research would conclude through analyzing the results from the usability test. This analysis would provide whether the design was successful or not. And through this the final research question would be solved which is to figure out the best AR concept for restaurant menu design.

(26)

5 User Research

5.1 User engagement and user experience analysis

The user engagement analysis was done through evaluating the data that were collected from a Finnish fast-food chain- Taco Bell Finland. Sales data about different sales channels for sales periods April 2020-June 2020 and April 2021-June 2021 were collected which are not public data but consent was collected to use them for this research.

Figure 1. Sales percentage taco bell restaurants April’20-June’20 (Virtanen 23 august 2021)

From figure 1, it can be found that almost every restaurant has most of the sales coming through self-service kiosk which is a payment gateway service provided by Openpos tech (OpenPos 2021). The second highest sales channel was front counter where the customers can order via communicating a human representative. While there was also substantial number of sales coming through online food delivery services Wolt and almost every restaurant has their lowest sales coming through another online food delivery service Foodora. This was during the first wave of pandemic when people were very conscious about the virus and hard restrictions were imposed by the government of Finland (Pilke 2020). After that there were changes in approach both from the restaurant owners and customers who soon saw the opportunity in delivery medium and utilized that.

0 10 20 30 40 50 60 70

Channelwise sales percentage of restaurants 2020

Self service Front counter Wolt Foodora

(27)

Figure 2. Sales percentage of different taco bell restaurants April’21-June’21 (Virtanen 23 august 2021)

From figure 2, it can be clearly seen that food delivery company Wolt has taken a big amount of share from the other channels and became the joint frontrunner among the sales channels alongside self-service kiosks. The growth of home delivery can also be realized from the growth of sales in Foodora; which is another delivery medium which has seen a sales growth of about 250%-300%. And the major decrease among the sales channels has occurred in the sales of front counter where customers used to order

through a human representative. With safe distance being imposed as a major element in hygiene and safety measures customers are more and more inclined towards home delivery and self-ordering which can be a big motivation for development of a 3D AR menu, which can be used through user’s smartphone.

Though this analysis was done only by utilizing analytical data from one restaurant chain of Finland, but it represents the overall engagement level as it was found that customers are preferring food delivery channels more than ever and this behavior is expected to grow (Statista 2021). Due to restrictions on movement, safety distancing, mandatory masks and hygiene concerns users are changing their behavior and industrialists are also changing their service designs and products that suits the needs of the customer while also utilizing the technological opportunities.

0 10 20 30 40 50 60

Channelwise sales percentage of restaurants 2021

Self service Front counter Wolt Foodora

(28)

While the user engagement provides with knowledge about user’s inclination towards self- ordering medium. User experience analysis would provide knowledge about the experiences that occurs during those engagements. For this purpose, customer feedbacks and reviews were taken into consideration to evaluate the matter. Sakib (22 September 2021) who is a manager of one of the taco bell branches and use to look after customer relation- ships, informed that most of the feedbacks with negative customer experiences are related to the appearance of the food. Customers complains about the food portion being too small, or the product is not as they imagined. Majority of customer dissatisfaction arises from Wolt and Foodora; where customers are not satisfied with the product due to lack of sufficient information about the product (Akdogan 24 September 2021). While these issues indicate lack of information in current restaurant menus but none of the manager believes the issues to be big enough to be alarmed about.

This tendency of user engagement towards self-service ordering mediums should be utilized and lack of information about the product which brings poor user experiences can be solved by designing a 3D AR based customer ordering channel where users can order from their place of choice with their smart device and order their favorite food while interacting with 3D images augmented in real world; this would provide them with information which must improve their experience about knowledge of the product beforehand, while ordering from a location which they find safe and hygienic will give them a superior experience than the existing.

5.2 User interviews- User preferences from AR menu

To understand user preferences and expectations from a restaurant menu, and to get user knowledge about AR and ideas towards an AR based restaurant menu a qualitative user study was done by using semi-structured interview method. Essential data were collected through these interviews which provides contextual knowledge about user preferences and system requirements for the new solution. This knowledge also helps to figure out the design features- e.g., interaction design and visual design. The study process started with developing a set of questions which would be basis of data extraction.

Throughout the interviews users were allowed to explore the matter using their imagination while always keeping them within the scope of the research.

The user profile is shared by the following table:

(29)

Table1. User profile for UX evaluation

No. Age Gender Nationality Smartphone type

Ordering medium

AR software usage

1 27 Male Somalian Android In person, web, self-service

Snapchat, Pokemon go, Instragram 2 26 Male Turkish Android Counter, web,

self-service kiosk

No

3 19 Female Finnish Android Front Counter Snapchat, Instragram 4 20 Female Finnish Iphone Self-service

kiosk

Snapchat, Instragram 5 17 Female Russian Iphone Self-services,

Web

instamasks

After greeting and welcoming, the goal was to make the interviewee felt comfortable by having a small talk with them. Interviewees were then informed about the reason and process of the interview, while explanations were provided about how data handling would be made. During the interviews the goal was to get data from diverse customer groups while emphasis was given towards younger generation as they are more intended to use AR.

There were total of 5 users that were interviewed during the study who visits restaurant and/or order foods online. While doing the interviews it was considered that diversity was maintained to get insights from different user group perspective. Interviews were con- ducted to explore the user preferences and choices in using a restaurant menu. While knowledge about user behavior and lifestyle were also gathered which should help to understand the design of the app. There was also information about the user’s exposure to AR which should give ideas about the amount of detailing possible in an AR app.

5.3 Data Analysis and Ideation

Based on the interviews of the restaurant visitors, different user has different factors that helps them decide their buying behavior. This information are valuable which provides valuable insights about the scope, features and aspects for the new AR menu design. It was intended to explore the user journey performing the steps of ordering and to explore the preferences that users have in such situation. The question that was asked from the interviewees about the information that are most important from a restaurant menu and their preferences. The information preferences are categorized by following table-

(30)

Table 2. Tasks matrix for food ordering process

User 1 User 2 User 3 User 4 User 5 Photos of Items

Catchy description Ingredients list Price

User reviews/

recommendations Language option Nutritional values Friends

recommendation

It is very clear that from above table that which are the most essential information that the interviewed group of users prefers to have them in their restaurant menu. Every single user would like to see a picture of the product to know how their product looks like and what it contains (ingredients list & Nutritional values). While all of them would also like to know beforehand the price they are needed to pay. A catchy description seems to be also a good information source about the product as it attracts the user to that product while providing them knowledge about what it contains. User reviews or recommendations also seem to be a valuable information source about a product while ordering. While some prefers to get recommendation of people, they personally know to choose their product. One user would also like to have the option of ordering them in their own language.

(31)

6 Prototyping and Design

The basis of this phase was the results from the research and knowledge about the UCD and UI principles. Firstly, a user scenario was developed to illustrate the possible user journey while ordering a food using the service (figure 3). This scenario was based upon the regular lifestyle of the interviewees which demonstrates the need for the app.

Figure 3. User scenario about the food ordering concept

A user journey map was drawn based on the user scenario and usual food ordering process (figure 4). This user journey helps to realize the touchpoints of the users with the app and helps designers to focus on stages that are more important to achieve the goal of the user. This user journey provides all the basic touchpoints, starting with opening the app and ending with order completion.

Alauddin is a student in Helsinki. Due to his busy schedule, he often eats out- side and sometimes orders food home. He likes to try new taste and explore different restaurants. But often he gets disappointed due to lack of information

about the portion size or a better understanding about the product. Recently Alauddin learned about an AR app “SEE-EAT” which allows him to get a 3D picture of the food and lots of other info that help him decide to buy a product.

He found out that the app is very easy to use. He just needs to scan a marker of the restaurant he wants to eat using his smartphone and it will then select and display the menu of that restaurant. The app augments -the 3D image of the menu items in the real space using his smartphone. It also provides the list

of ingredients, price, nutritional values, and his friends reviews in the same view, depending on his choices. Alauddin can now easily explore new places

with the help of “SEE-EAT” app, with an improved experience.

(32)

Figure 4. User journey map for food ordering process of “SEE-EAT” app

Based on the user journey map and the scenario a Low-fi prototype was developed (figure 5). The low-fi prototype brings the most fundamental options of this app and allowing the user to successfully order their desired food with the augmentation of 3D image of the product. While developing this prototype the goal was to implement only the basic func- tions and there wasn’t much emphasis given on color choices, details, or interactivity of the design.

(33)

Figure 5. Low-Fi prototype design for the Augmented reality concept

The Low-Fi prototype was later used to develop the Hi-Fi prototype design. To design the Hi-Fi prototype, the more emphasis was given to the details of the design using the UI principles and UCD methods. Detailing was also done utilizing the information collected during research about the user needs, expectations, and frustrations.

Color psychology is a large discipline which deals with how different color are related to human emotions. Eiseman (2017, Ch7), associates color psychology to nature by illustrating how yellow color can be related to sun which brings warmth and joy. While blue is to sky, that brings the feeling of freedom. To design the hi-fi prototype the color palette was chosen to provide the users with knowledge, passion, wisdom while also provide the feeling of liveliness and freshness. There was also effort given to provide some sense of freedom, imagination and inspiration while choosing the colors for the design. So, the intent was to choose some colors combinations that are related to yellow, blue and green (figure 6).

Product cate- gory screen to let the user choose the product

User prompt. to rotate the screen into vertical

Edit selected items to modify them as the choice of the user

Confirmation screen to make sure order is right The selected

burger in augmented in the table of restaurant, with details about the product.

(e.g., ingredients, price)

(34)

Figure 6. Color palette choices for “SEE-EAT” AR concept design

While designing the visual part of the design, emphasis was given also to provide icons that are familiar to the users. It was also emphasized to provide conventional naming so that user feels familiar and comfortable environment (figure 7). Having familiar icons and naming provides users with emotional attachment with the design.

Figure 7. Choice of Icons and naming for the conceptual design

While designing the prototype similar informative items were grouped and color coded for ease in browsing the design for the user. The color coding was done in a way that allows

(35)

users to differentiate design components in terms of amount of use. While the more frequent used components are highlighted in a way so it can be easily visible, the less fre- quently used components are less highlighted. Highlighting was done both by color choices as well as change in opacity (figure 8).

Figure 8. Use of contrast and opacity for better visual experience

While designing the most central part of the concept which was the augmentation of 3D elements, careful consideration was made to allow the user to distinguish the virtual components from real components. It was found that apart from the 3D image of the food item, there are also other components which provides related information for the food. Each of those items were designed in such a way that they don’t obstruct the visibility of the user in the real world, while providing the necessary information of the virtual world (figure 9).

(36)

Figure 9. 3D menu augmentation concept with relevant information

Knowledges from the research were used to prepare the concept of 3D menu design. To design the concept of the screen where the augmentation will occur, results from qualitative interviews were used which provided the user needs and expectations. Utilizing those information and other design choices mentioned in the previous sections, the 3D menu augmentation concept screen was designed. Adobe XD was used as the development platform where the prototype was developed. During the design, knowledge from task matrix were used to include different elements. For example, it was found that every single user looks for ingredients list and price while buying a product. So, the design was such that this information is provided right over the 3D image of the food and user can simultaneously observe both information (figure 9). While more than 50% of the user also finds reviews and information about nutritional values to be important factors while deciding to buy a product, this information is provided in two sides of the product image. The concept was designed in a way that the system will have links to social media, which will allow users to communicate with their friends by providing reviews about their experience with cer- tain product. In the information architecture of the design, nutritional values and reviews are grouped as secondary compared to the ingredients and price list, so there are choices provided for the users whether to view this information or not. Users can minimize or max- imize the lists based on their requirement (figure 10).

(37)

Figure 10. Information architecture of 3D menu design

Throughout the design effort was made to make the design simple to use while bringing in every aspect that the design solution is supposed to provide. Based on Galitz theory, The visual aspects such as balance, symmetry, proportion, and economy were considered and adapted to the design. While factors such as consistency, user control, flexibility, error prevention, recognition etc. were also considered and utilized to the design. As the design was conceptualized for AR technology, the technology itself is new and there has not been any standardized design methods yet, but formulas such as transparency or opacity were adapted to the design that are being used in other AR applications.

An Augmented reality concept for a restaurant’s menu based on existing user experience data