4.10 Non-Functional Requirements
4.10.2 Safety Requirements
• Safety instructions for the Nokia N770 and P3-DX should be followed ac-cordingly, those are detailed in the operating manuals .
• The RTMU operator should be familiar with the IT department safety instruc-tions, including following strictly lab practice rules.
• When using RTMU application for teleoperating tasks it is advised to use small environment map areas.
• Tests should be carried out with less distractions and should not happen in busy times to prevent accidents or unnecessary delays.
• The lab assistant should be easily reachable and if possible present on the site of teleoperation testing.
It is useful to mock the expected interface in wire frames because it illustrates major input / output elements and their positions, helps prototype the end product, it can also identifies any UI shortcoming or alerts about usability practice in addition it may offer ideas for additional features good to have. The wireframes can undergo changes but retain similar form.
On the Maemo 2.2 Hildon is used to develope GUI applications, it offers different usable view areas. When RTMU is launched it is expected to occupy the normal application view area shown in Fig. 48a, the full screen Fig. 48b can be activated by pressing full screen hardware button on the Nokia N770. The full screen provides more viewable area and in practice it will contain the environment map.
Functional buttons and feedback labels are placed on the toolbar shown in Fig. 49, the toolbar can be toggled visible or hidden using a menu item and the operator can close the visible toolbar at anytime by touching the X "close" toolbar button.
The top-right numbers in Fig. 49 represent the available height in different screen views, a possible maximum height of 360 pixels is available for the map or image display in normal view with a visible toolbar. If the fullscreen is activated with visible toolbar then 420 pixels can be utilized to draw the environment map or image information.
(a) Normal 672x396 pixels view
(b) Full 800x480 pixels view Figure 48. Hildon Application Views
Figure 49. Single Toolbar, 360 or 420 pixels
The main menu items are displayed in Fig. 50 the settings menu item when activated popups a dialog shown in Fig. 51, each performs a purpose indicated by its caption and the behaviour is detailed in the later subsections.
Figure 50. RTMU Menu Items.
Figure 51. RTMU Settings Dialog.
Fig. 52 represents banner popups, any notification to the operator is displayed in a popup banner that timeout and hides depicted in Fig. 52a. Fig. 52b represents the robot speed velocity as a progress bar and is activated by pressing the hardware buttons ( + , - ) on the Nokia N770.
(a) RTMU Notification Message Banner (b) RTMU Speed velocity Indicator.
Figure 52. RTMU Auxiliarly Popup’s
accepts commands from RTMU client and if commands are applicable then they executed by the robot HW. The command outcome and robot status are retrieved by ARIA server and consequently sent back to the RTMU client output call back handlers.
Figure 53. RTMU Deployment Diagram.
5.3 Architectural Design
RTMU was developed using MVC pattern visible in the class view presented in Fig.
54. The MVC pattern is used to design the structure with only three main classes along with other structures for support and code organizing.
Figure 54. RTMU Class View.
ArMap object which encapsulates the data con-tained in a .map file and constructs the map by iterating through the lines and map objects.
UI The operator views RTMU from this class, the view provides elements to send commands to the controller object. The class provides all nec-essary initializations, creation of UI widget’s and listens to certain signals e.g. exit RTMU are implemented. Other signals are called back to the registering controller object. It also con-sists of the stage object and provides methods to manage the settings dialog.
UI.h
Controller Provides a control layer between the robot par-ticulars and the UI representation. It is com-posed of a Player and a UI objects. The main task is to update the view according to user command activation and send them to the Player object. The Player object then calls back controller methods which can update the UI ele-ments directly or by calling UI object methods.
Controller.h
Player Encapsulates all robot related activity with the help of ArClient ARIA client object, it is used by the controller object where informa-tion flows both ways. This class is built us-ing multiple ARIA examples and implementa-tion to handle map and image requesting. It is designed with expansibility in mind where new commands can easily be realized.
Player.h
Settings Provides a mechanism to store application set-tings mostly are ARIA server connection pa-rameters by employing a GConfClient object.
Utilities.h
5.5 Sequence Diagrams
SD ( Sequence Diagrams ) are used primarily to design, document and validate the architecture, interfaces and logic of the system by describing the sequence of actions that need to be performed to complete a task or a scenario [47]. Fig. 55 covers the connect SD and shows the initial and major communication that occurs between RTMU and ARIA server.
Figure 55. Connect SD
Figure 56. Move or Stop Command
The next SD in Fig. 57 illustrates RTMU operator requests a robot image. It can be noted that the image is requested by pressing the Nokia N770 hardware Escape key the first time, if the image is displayed then a second press would return the normal map or robot view.
Figure 57. Request Robot Image
Move Backward
RTMU application was developed on a Windows 7 machine with the use of Sun VirtualBox for virtualizing Debian distribution of Linux. Installing the environment was a time consuming but fairly straight forward task, it also involved different shell commands to adjust the environment when needed.
The Nokia N770 also went through patching with a later Maemo OS called Maemo Hackers Edition, mainly because this release offers more developer oriented plat-form and root privileges are easier to gain than the original manufacturer release.
RTMU was developed with the following tools:
• ARIA 2.7 C++ SDK. The source code is free and was recompiled for the N770 armel target with no changes to source code.
• MobileSim Robot Simulator was used as the main robot testing tool.
• Maemo SDK and rootstrap v2.2 for i386 and armel with C/C++ as program-ming language.
• Scratchbox 1.0.17 and toolchains cs2005q3.2 and arm.
• Xephyr and Xming for XWindows system, TotalCommander for ssh and file management.
• Slickedit 2010 as developement IDE, VncViewer for grabbing the N770 screen.
• MiKTeX 2.8, Microsoft Visio 2010, Sparex Enterprise Architect, PDF-Exchange and Sumatra pdf viewer, Doxygen for commenting and documentation.
The N770 emulator was used at the initial stages of development but less frequently when RTMU implementation matured and required faster response for map updates.
The ARIA source code had to be recompiled under scratchbox for both i386 and armel rootstrap, each produced .so (Shared Object) files that RTMU requires in order to communicate with the robot.
Function pointers and callbacks are extensively used by ARIA classes. In fact RTMU output functions use those callbacks for displaying all feedback information ( Map, Image and Robot Status ) they proved to be reliable but demanded threading and caused slow display updates due to the limited N770 resources. The emulator was not used when all code parts have been enabled i.e. movement commands and updating the robot position, this shortcoming maybe not present in newer Maemo versions but worth investigating.
Hildon UI with its GTK+ extension was used to construct the GUI layout. The Maemo 2.2 SDK is outdated and finding information and examples was time con-suming adding to the fact coming from different UI development made the learning curve steep but eventually recognized the trend and became easier.
Figure 58. RTMU Startup and Idle State
5.8.2 RTMU Menu And Settings Dialog
(a) RTMU Menu (b) RTMU Settings Dialog Figure 59. Menu And Settings Dialog
5.8.3 Teleoperation With Environment Map
When RTMU connects to an ARIA server that offers environment map the outcome example is in Fig. 60. The operator can adjust the map zoom size using the combo dropdown currently set to 60% and view a larger map area, the robot movements is updated on the map.
Figure 60. Connected to ARIA Server with map
5.8.4 Teleoperation With No Map
When RTMU connects to an ARIA server that does not offer environment map the outcome is a blank map area with a circle that represents the robot and a white radius line is updated to reflect the robot header angle as in Fig. 61 the banner times out and becomes invisible.
Figure 61. Connected to ARIA server with no map
Figure 62. RTMU connected to ARIA Server with map and hidden Toolbar.
5.8.6 RTMU Ratio Increase / Decrease Indicator
The robot speed velocity can be increased or decreased by pressing the ( + or - ) hardware buttons on the N770, Fig. 63 displays a progress banner with current ve-locity value, if the operator reaches either maximum or minimum value then another banner stating value cannot be adjusted and the operator must use the opposites but-ton.
Figure 63. RTMU connected to ARIA Server and velocity banner visible.
5.8.7 RTMU Robot Image Feedback
If the ARIA server does not support environment image requests then the operator will be notified otherwise and the environment image will be displayed as an exam-ple Fig. 64, this command is issued when the operator pressed the Escape hardware button on the Nokia N770. If image is displayed then pressing the Escape button again returns to the map or robot view.
Figure 64. RTMU connected to ARIA Server and Requested Image.
5.9 Installation and Help system
Typically a Debian package file is the preferred method to install any new applica-tions on the Nokia N770, but due to time limitaapplica-tions 4 files related to RTMU needs to be copied manually commonly under a directory in the user home path.
Required Files:
• RTMU executable.
• ARIA shared library files : libAria.so and libArNetworking.so
• RTMU logo file RTMU_logo.jpg
There is no interactive help for RTMU but there are instructions and HTML infor-mation on how to successfully run RTMU from the provided file kit.
performance and suggests GUI design experience feedback. Each use cases pre-sented in Chapter 4 are treated as a single test case in addition to added features to utilize maximum display and regulate robot movement speed as can be seen in Table 8 with results.
6.2 Teleoperation Testing
Teleoperation experiments were necessary to evaluate the RTMU application and will further confirm the functional test results. It also offers a real experience and more understanding of the teleoperation nature on the field, in addition to applica-tion consideraapplica-tion for further development. The test arena environment is depicted in Fig. 65 with dimensions in meters and rough obstacle and robot home/goal posi-tions. It was not possible to use the teleoperating map presented in Chapter 4 as per required due to the battery limitation on ARIA server laptop.
6.2.1 Navigation Test
The operator was with the robot in the test arena and did not require environment images or map to navigate the robot from the home location to the goal area. Teleop-eration time took 1 minute and 10 seconds, for returning the robot to home location required 1 minute and 20 seconds, no collisions occurred. Removing the obstacles between goal and home position required only 40 seconds to complete.
6.2.2 Navigation and Exploration Test
The operator was outside the test arena where the robot was completely invisible and in another room, only teleoperating the robot from home to goal area was required.
The operator relied heavily on requesting environment images from the robot cam to explore and then navigate the robot between obstacles. The operator successfully reached the goal area within 4 minutes and 15 seconds.
Table 8. Functional Testing Results
Use Case Test MobileSim P3-DX Robot
Comments
Get Robot Status • • WLAN signal strength
was not implemented thus not tested.
Get Environment Map • •
Map Zoom in/out • • Avoided 100% for large
maps otherwise will take longer time to display and may crash the application.
Toggle Toolbar / Full Screen
• • Added feature for robot
status and employing full screen for map and image.
Move Robot • •
Increase/Decrease robot speed
• • Added feature for move
robot test case
Figure 65. Teleoperation Test Arena
6.2.3 Robot and Test Arena Images
The image on Fig. 66 shows the actual robot in the test arena where the robot is located near the home position ready for commands by the RTMU operator. Fig. 67 shows the robot passing the obstacles and moving towards the goal location. The laptop running ARIA server is wired for power supply and are carefully displaced when robot is moving without affecting the teleoperation experiment.
Figure 66. Robot Home Position
Figure 67. Robot Goal Position
6.3 Operator Feedback
The operator required small amount of training to use RTMU and reported its ease of use and GUI understanding, testing the functionality was a straight forward prac-tice except the need to restart RTMU when disconnecting from ARIA server. For teleoperation activity the following items were noted:
• Request for turning the robot at particular angle rather than keep pressing the turn buttons constantly especially for rotating the robot in opposite direction.
• Experienced delay in image requesting and this explains the longer duration for exploration test.
• The operator suggested live video feedback when there is no map available.
• Teleoperation task is easier when operator got familiar with the environment.
• Robot safe driving is a must have when only relying on image feedbacks for map less exploration.
• RTMU was successful for teleoperating P3-DX in the small environment.
• The operator inquired the possibility of using RTMU on newer Maemo de-vices.
tion in the disconnect function.
WLAN signal strength value feedback label was not implemented due to two main reasons: WLAN connectivity on the ARIA server laptop was constantly discon-necting from network and the other reason was an extra toolbar is needed which was not practical to introduce on its own. There was an idea to add a toolbar with move command options but implementation time will be increased and this adds more testing and tweaking in general.
The WLAN feedback information is necessary when teleoperating the robot within a wide environment, it can be represented as a direct value label or can be a warn-ing message or a sound alert when weak level value detected. The benefit of this information is to prevent the robot from entering an out of scope "communication"
area and strand the robot completely disconnected from the operator.
Running a different ARIA server version resulted in an unexpected event for re-questing the map, the same command was supported but encoded differently. The actual map file is in text format but when saved the contents appeared in binary format. Possible problems can be related to different compiled platform encoding, or an outdated version of ARIA server sends map in special format. An attempt at the test site to save the incoming map as binary similar to handling images requests was not successful because other file IO operations were necessary to change, thus delaying allocated test time.
From results it is evident time required to complete navigation and exploration test was larger than navigation. The main cause of this difference is mainly related to the lack of environment map and the frequent images request increased delay.
Therefore the environment map is necessary for teleoperation tasks when there is no direct vision of the robot. Image requests should be reduced to a minimum since
this adds communication and reduces performance.
MobileSim simulator was highly valuable for developing RTMU implementation and it preciously reproduced the actual robot teleoperating behaviour, some local-ization errors were experienced and on the RTMU robot the robot actual home po-sition was not correct and at times crossed over map lines giving false indications of robot position. This suggests care should be increased in dangerous environment locations and introduce different localization algorithms.
The full screen on the N770 provided more display area but zooming proved more effective, the only limitation is its high processing demand not practical for large maps. A work around would be achieved by partitioning the map into zones and load only the robot zone area with extended area surroundings, this will increase availability and less hardware demand. Zooming at large values e.g. 70% and over should use map partitioning technique.
Several teleoperation sessions established between RTMU and P3-DX robot have shown a stable response and performed successfully the experiments which con-firms the reliability of the implementation. Although more testing is required for larger environment area and by different operators. It is anticipated to obtain simi-lar results and findings relate to previous study outcomes and experiences. The next final chapter concludes this thesis and suggests further development.
lines and for test ideas to evaluate RTMU teleoperation performance. The experi-ments conducted and results indicate reliable quality to utilize the system for vis-ible robot navigation. As for exploration tasks further work is needed and can be achieved by adding a live video streaming feedback. This is only required when no environment map is present.
Setting up an easy to use development environment was time consuming but in-creased productivity at later stages, from SDK installations to configuring network connections between mobile device and development PC was a learning experience.
Testing the implementation with different ARIA servers produced unexpected out-comes. RTMU was designed to handle such events and attempt recovery but the major show stopper was caused by how the same command outputs in different en-coding result, therefore it is best to use a single server for both development and for testing live on the actual robot hardware.
Robot ARIA SDK library proved to be reliable and easy to port on different plat-forms such as on the Maemo version 2.2 used in this thesis, it is expected to work as efficiently on newer versions. The implementation logic can be recompiled but the UI class will need to be updated to use different API’s or widget’s and is expected to offer more thread safety and increased display response time.
The MVC architecture pattern used in this thesis for UI and robot logic separation is recommended for implementations mainly because development can be accelerated and its independence design methodology encourages improvements. In this thesis some callback functions were not combined within the class itself because it was how Hildon UI library is designed to call signal routines.
The Maemo 2.2 emulator shortcoming were related to threading and UI update.
Thus an actual device was used to test map drawing and process real time call backs. Developing RTMU using emulator was slow and in fact some features were
dropped because they added delay and caused deadlocks, sometimes recovering from bad code blocks required environment reset and at times rebooting the devel-opment machine.
The most extensive effort was put in the GUI presentation layer, where map drawing involved calling basic line functions which is not suitable for large scale views or
The most extensive effort was put in the GUI presentation layer, where map drawing involved calling basic line functions which is not suitable for large scale views or