In this section, we present the main phases in the research process, including the focus and outcome of each phase. We also mention the connection of these phases, outcome of one phase can be used as income of the next phase.
Problem Identification
During this phase, we investigate the state of the art (e.g., literature review) concern-ing two fields of research: home automation and user context, existconcern-ing issues as well as available solutions and implementations. It is discovered that both research fields attract high attention and quite a lot of applications. Among the cases, the most popular ways of using user data are from home integrated sensors and activity/state recognition from sensor-level data, such as motion detector, smartphone accelerator. However, a joint res-olution of interpretation from high-level user data (not directly coming from sensor level) and practical implementation in HAS is somehow missing. Our research approach starts with defining and understanding what problems we are trying to solve, thus, looking for a meaningful solution. From these findings, we identify our research scoping and establish the context where the inquiry should focus on. The output of this phase is a clearly defined research context - a "proposal" which will then be used as the input for building a concept modal of such integration. At the end of this phase, research questions and delimitation have been clearly defined to reflect our research goals, as described in the Introduction section.
Conceptualization
The Conceptualization phase immediately follows the proposal and is intimately con-nected. The idea is developed based on the awareness of our identified problem. A possi-ble outcome of this phase is a tentative design where we select user attributes for further analyzing. This conceptual model is the early stage of a system design that we will then set up and implement. This phase is important for transitioning requirements into sys-tematic descriptions. Moreover, some tentative ideas become obsolete and new concepts evolve during this stage after primitive analyzing. We also investigate and define system specifications during this phase, which set a scope for system design.
Design and Development
The Tentative Design is further developed and implemented in this phase after investing a considerate amount of effort in assessing related ideas. To make sure the proposed solution is technically possible in a specific use case, we explore different HAS platforms in term of context integration supported features to select the most suitable items for a system implementing. An abstract architecture to integrate user context into HAS is carefully designed. As part of this phase, we also develop an implementation applying the architecture within the German context. "Design and development" is usually considered the most critical phase where practical aspects of our presumptions are verified, and it also forms a premise to evaluate the efficiency of the proposed solution.
Evaluation
Once constructed, the architecture we proposed needs to be evaluated according to "cri-teria that are always implicit and frequently made explicit in the "Problem Identification"
phase" (Vaishnavi, Kuechler, and Petter, n.d.). To achieve our listed research goals, we decided there is no better way to evaluate an improvement by comparing with existing solutions. Due to the nature of HAS, the scenario attributes, including user relevant at-tributes such as living condition, social standards, etc., of the installed HAS, affects its efficiency in multiple ways. As an outcome of this phase, we carried out a heuristic mea-surement in terms of energy efficiency and compared with measures of a typical German use case (Sarmento et al., 2017). Regarding user adaption aspect, we conduct a survey to evaluate user perception when it comes to integrating their privacy and willingness to adapt to such systems.
Conclusion
The conclusion phase marks a milestone of our research. We review the results deriving from the study and validate the revised theoretical base. By examining the work and verify all research goals, we also discuss the contributions of this study, future research potential and the quality of our solution in different aspects: technical, sustainability and environmental.
This phase of the research focuses on defining system specification and use case, hence build up the possible system architecture. We investigate automation platforms, user con-text dimensions, and ways to integrate them into an overall structure where concon-textual data is efficiently leveraged. This section is thus organized in the following order to describe the process: system specification, technology stack, system architecture, and application development.
4.1 System Specification - German Use case
To verify the practical aspect of the proposed user-context enhanced smart home system, we need to define specific parameters and metric base as a foundation to implement, thus, validate system usability. The study leverage available study on the fixed scheduling model in HAS to compare upon, more precisely - German use case. In this section, we present specifications of a typical schedule according to the German lifestyle and the scenario where our implementation is built upon. This task aims to scope and expose a clear view of the referenced use case, hence provide scoping for system evaluation afterwards.
Inspired by (Sangogboye, O. Droegehorn, and Porras, 2016) study, we present require-ment specification adapted to a single-occupant apartrequire-ment. Activities are scheduled based on a weekly basis. The requirements for smart home strategy are influenced by user be-haviors and user expectations from the system. As a result, HAS is expected to maintain system efficiency in term of energy usage and ensure liable level of user comfort at the same time. To simplify our scenario, we don’t consider in-house user recognition and motion detection strategies.
User requirements
• User wants to have full control over the smart home system where he/she is, whether at home or away.
• The automation system should provide a comfortable living conditions when user is at home.
• The system helps to manage the energy usage of home appliances in an efficient way.
Fixed-scheduled specification
• Occupant leaves and arrives home following a schedule during weekdays (5days/week).
• House will be fully occupied during weekends (2days/week).
• Occupant leaves home at 8:00 and arrives at 17:00. The heating system is expected to start heating/cooling at least 30 minutes before occupant gets home, which is at 16:30.
• All lights are expected to turn on at sunset only if user is at home.
• Occupant goes to sleep at 23:00, all lights are turned off after this time.
The system aims at analyzing gathered user-context data, thus, react more semantically meaningful to out of the ordinary user behaviors. For this enhanced strategy, we consider the following context-enriched user scenario.
Context-enriched scenario
• System has access to occupant’s calendars and events, including: start and end time of an event, location of an event.
• User has wearable tracking devices with contextual information: activity mode, heart-rate, sleeping mode.
• Location tracking is enabled from user’s device.
Context-enriched specification
• During weekdays, if user has any event scheduled, system should estimate time to arrive home from the event location and adjust the devices switching on/off automa-tion accordingly.
• During weekends, for any extra activities, recalculate when occupant is at home and adapt.
• Start heating/cooling system at least 30 minutes before occupant gets home, adapted to scheduled events if any.
• All lights are expected to turn on at sunset only if user is at home.
• Turn lights off at sleeping time, using more accurate sleeping state from tracking device.
User Comfort Requirements
• In-house temperature and humidity level should be maintained at a comfortable level.
• Energy usage and improvement from automation information should be easily ac-cessible.
• A convenient level of control over all devices at home.
4.2 Technology Stack
Before diving into the system architecture and implementation, in this section, we present a clear overview of the technology stack that has been utilized. It is essential for system development to pick the right combination of underlying tools and technologies. We leverage Home Assistant as the primary platform for development, along with smart home devices (e.g., lights, thermostat), sensors, smartphone, and wearable device. Contextual data sources such as calendar, health, location are included as part of the system and are discussed with more details in the System Architect section. Figure 4 presents the combination of technologies that have been studied throughout "System Development"
phase, comprised of programming language, platform, network infrastructure setup, and software underneath.
Home Assistant
Home Assistant is an open-source home automation platform built and run from Python, considered the world’s biggest open-source home automation platform with an active and strong community of developers and users. Home Assistant provides three main modules (see Figure 5) to support smart home system: Home Control, Home Automation and Smart Home.
• Home Control, which is supported by Home Assistant Core, is responsible for col-lecting all information and controlling connected devices. Home Control plays the role of a communicating gateway between devices and automation strategy.
Figure 4:Technology Stack.
• Home Automation triggers commands based on information transported within the system and user configurations.
• Smart Home refers to the smart handler which triggers commands based on past behavior.
Figure 5explains the full picture of how different factors fit in Home Assistant platform.
User interacts with the system through a user interface and can initialize custom con-figurations, while different modules interchange information back and forth with each other using standardized data structure and set of commands. Home Assistant Core is the primary channel where smart strategy is handled, made possible by four parts: State Ma-chine, Event Bus, Service Registry, and Timer. Each entity supported by Home Assistant (e.g., devices, sensors, external services) has its state and attributes. State - the critical piece of information that links devices with the controller - is managed byState Machine, which keeps track of the states of things and fires a "state_changed" event when a state has been changed. Event Bus - the beating heart of Home Assistant facilitates the firing and listening of events. WhileTimer sends a "time_changed" event every second to the event bus, the Service Registry listens on the event bus for any "call_service" event and also allows external services to be registered. This structure also implies the event-driven
Figure 5:Home Assistant Architecture.
characteristic of Home Assistant, as defined in its documentation (Home Assistant, 2019)
"The Home Assistant core is event-driven. This means that everything that happens is represented as an event: a light being turned on, a motion sensor being tripped or an automation triggered. Each event has an attached context. The context can be used to identify which events have been triggered as a response to other events, which user triggered the original event and with which authentication."
Dynamic DNS and Port Forwarding
Initially, Home Assistant is installed on Hass.io and is only available within the inter-nal network. Working Home Assistant server from the local network poses no issue;
however, it is inaccessible from the outside world, which, in our connected society, is
non-neglectable. To open up a remote connection to the server for development flexibil-ity, we apply dynamic DNS and port forwarding on the router to make the home server accessible behind NAT. This technique is made even simpler with Home Assistant native component - DuckDNS - a free dynamic DNS service that allows us to point a subdo-main under.duckdns.orgat our home assistant server, more specifically, we can map the private IP address of the home assistant server to a free-of-charge.duckdns.orgdomain.
After having a public domain, we need to change the router setting to allow external re-quests to traverse through the protected network. Port Forwarding is the technique in computer networking that allows remote nodes to connect to specific nodes (computers) or server behind LAN.
Figure 6:Network port-forwarding setup.
Figure 6demonstrates the network setup to allow the home assistant server to be accessi-ble from outside the network. Supported by DuckDNS, we can register a free domain and map this domain to our public IP address. As a result, the home assistant server can be visited from any computer with internet connection via the registered domain, and it will point to the server standing behind NAT. This setup simplifies the developing process, and it is more likely to be a realistic setup where a user can remotely interact with the server without having to be in the same network.
Frontend Platform Services
Home Assistant comes with a powerful frontend tools to develop client application for user to control and monitor smart home system remotely. The frontend is built with Poly-mer - a web development toolbox that provides modern libraries and handy components.
Our approach configure frontend application on top of this layer using configuration sup-ported by Home Assistant - in yaml format. The control flow is also implemented on top of this platform, every logic is set up through yaml configuration.
4.3 The Proposition
According to the research motivation that was described in the previous section, regard-less of the fact that studies have shown possible solutions to build a HAS using different technologies, sensors, and smart devices, an overall integration with user-centric con-textual data is still missing. Thus, in this section, we propose an overall architecture to integrate the actual context of users into HAS in a sound way that can benefit from con-textual data and the ability to automate energy monitoring process provided from a home automation platform. The proposed architecture is inspired by a standardized design for development and implementation of future smart home technologies (Cook et al., 2013) which enable remarkable easy-to-install features. An overview of the overall architec-ture is shown inFigure 7. The infrastructure substantially consists of four main sections:
Physical Appliances, IoT Hub, Context Builder, and Reaction Dictionary.
Physical devices refer to home appliances (e.g., heater, refrigerator, ventilation controller, air conditioner, etc.), smart lightbulbs, smart meters (e.g., thermostat) and sensors (e.g., motion detector) installed in a HAS that are connected to a controller through various protocols, for instance, ZigBee, Bluetooth, Wifi, and so on. The connected devices and data from sensors are centrally managed and monitored by an IoT hub. There are quite many different platforms that support these functionalities. Each of the HAS platforms has its structure and protocol to manage connections to external devices and services. The most common method is through API and HTTP protocol.
A typical structure of a HAS platform should support a user interface for the user to inter-act with the system, including monitoring and remote controlling devices. Apart from pre-defined UI provided from the platform, most HAS platforms support external application building through API. Another vital component within an IoT hub is the scheduler. The scheduler refers to user-defined events to switch on and off devices at a specific time usu-ally based on daily, weekly, or monthly schedule and depends on the living environment.
For example, the scheduled controller establishes comfort temperatures (standard-based) during the expected home usage schedule. It is common in office buildings, or even in
Figure 7: An overall architecture of user-context integrated home au-tomation system.
houses where inhabitants do not want to be bothered with or usually neglect the manual heating/cooling adjustment. According to (Felix Iglesias Vázquez et al., 2011), the en-ergy performance is generally rather weak, but comfort ratings are satisfactory provided that people are at home during the scheduled time.
Context Builder component is in charge of processing high-level data from accessible contextual user data. The output of this module can be knowledge which the designer uses to decide corresponding reactions or adaptions. Besides predefined knowledge base (built upon surveys or specific use case of a particular country, region or neighborhood), machine learning algorithms such as classification, fuzzy prediction are fit in this module to provide more meaningful knowledge out of low-level sensor data. Separating the con-textual builder to be a dependent functional module enhance flexibility and scalability. It is thus auspicious to apply different techniques and achieve more meaningful information out of the same data set.
However, the extracted information from different sources can be in various format which produces obstacles in finding a way to integrate as a consolidated system. It is critical to have a unified format of the output in which the context-enhanced system recognize
and adapt without any confusion. The nature of smart home is automated process on controlling operation mode of devices based on a set of conditions, known as a rule-based activity. A rule should contain adequate information to tell the system what to do and when to do something meaningful. Such rule appears in this form:
I f < c o n d i t i o n s > t h e n < t a k e _ a c t i o n s >
Reaction Dictionary contains a set of rules in the above format. Actions are organized corresponding to specific user context. In this design, the dictionary can be built inde-pendently, which brings the benefits of further extensions or capability to integrate into other systems. This knowledge structure is organized as a rule-base dictionary, in the form ofA→B, whereAis context-aware conditions andBis system reaction. In the con-cept model, we separate this module and the Context Builder module due to the purpose of use, however, in the implementation phase, the reaction dictionary can be considered the output of this context builder as they could be closely connected in term of technical implementation.
Most of the home automation platform has strong support for implementing automation scripts following their specific instructions. Home Assistant is built upon Python, thus offers all powerful tools that Python has to offer. With current support, there is, unfortu-nately, no platform-independent way to implement smart strategy for a HAS. What these platforms support in common are multiple triggering techniques. The most common trig-gers are time-based, location-based, and event-based. All these activities happen within the IoT Hub component, where the devices should be connected and made controllable under supervision of the automated system. With the support of IoT platform, connecting multiple smart devices are doable regardless of communication protocols. The most used protocols in the market that can be mentioned are ZigBee, Bluetooth, and HomeMatic.
Follow this proposed architecture, we develop an overall smart home system where actual user context is analyzed and integrated as part of the system, forming a fully functional HAS. The implementation adapts to single-user scenario within German use case with specifications described in the previous section.