This is a Bachelor of Science study on a graphical test campaign parameter editor for a Testing and Test Control Notation 3 (TTCN-3) development environment. This work is a design-oriented research project, that is, a constructive research project [1], and has been made as an internal project at OpenTTCN Oy. The primary research method used in this study is literature survey.
TTCN-3 is a test specification language standardized by the European Telecommunications Standards Institute (ETSI). Its primary field of application is black box conformance testing of telecommunication protocols such as those found in cellular network base stations.
Conformance testing means verification that a system complies with a specified standard.
Conformance testing is usually performed to certify a system as standards compliant. Some of the most well-known fields of application for TTCN-3 are in the domain of protocol conformance. Some example domains are testing of 3rd Generation (3G) mobile network base stations and Long Term Evolution (LTE) base stations and handsets. Another field of applications is wireless network base stations such as Worldwide Interoperability for Microwave Access (WiMAX). It can be applied to black box testing in other domains as well.
The objective of this study is the design of a graphical parameter editor for test campaign parameters. The framework for the design is OpenTTCN IDE, a graphical TTCN-3 test script development application built atop the Eclipse platform. Eclipse is an open-source integrated development environment. It can also be used as a platform for building development environments for new languages. The test compilation and execution components are based on OpenTTCN Tester for TTCN-3, which is used for building test systems based on TTCN-3 for specific application domains.
1.1. Background
Test suite parameterization is the definition of important options for a specific execution of the test campaign. In the context of TTCN-3 this is called module parameterization. An automated test system would want to support different device configurations, such as different transport mediums or optional features, as well as different hardware configurations. Without test parameterization this would require different test suites, which are usually certified. This would create unreasonable monetary overhead for specification of test suites and make it difficult to standardize them, which is why the TTCN-3 standard defines test suite parameters.
The TTCN-3 standard defines the use of test parameters, but it leaves the provisioning of the test parameter up to the implementation. Traditionally these parameters have been stored in some kind of file format in the test management application and then provided to the executable test campaign via a standardized Test Control Interface – Test Management (TCI-TM) Application Programming Interface (API).
In the OpenTTCN Tester for TTCN-3 application module parameters can also be defined in a file similar in format to TTCN-3 test script and associated directly with a testing session.
The problem for new users of TTCN-3 or even experienced users getting familiar with a new test suite is the large amount of test parameters test suites define. A modern test suite can contain dozens of parameters, which are defined in the TTCN-3 test script. These parameters can have varying types, such as integer, record or enumeration. A record type is a structured type similar to the struct type in C-language.
Without a graphical editor for parameters, a user new to a test suite would have to dig through the TTCN-3 source code to find what parameters there are and which values are acceptable for each parameter. A graphical parameter editor can do this work for the user, gathering information about available parameters from the test suite and what are acceptable values for them and displaying this information in an easy to digest form with a graphical user interface.
1.2. Objectives and limitations
The objective of this study is to design an easy to use test suite parameter editor for the OpenTTCN IDE development environment. This study will survey existing practice in defining test parameters and existing works on usability applicable to the project.
The intent is to make a graphical user interface that can present the parameter information in a fashion suitable for human consumption and modification. The editor will show a list of possible parameters for a compiled test suite and the user can edit them directly. Editing of integer values in hex, octal or binary formats would be allowed via a custom editor.
Enumeration parameters can show the acceptable list of enumeration values as well as their documentation. Complex types such as records could be shown as a tree structure with the individual elements editable with the editor components suitable for them.
The actual implementation work is left outside the scope of this paper due to scheduling constraints. The exact interface with the compiled test repository and the physical storage format are also left outside the scope of this paper, it is simply assumed the required information can be obtained in some fashion from the other parts of the system. To have some basis for the editor, the kind of information required will be explained.
The result of this study will be a thesis detailing the findings of the literature survey and the design of the parameter editor. Some internal documentation for OpenTTCN Oy use has also been produced and commented upon in the thesis, but as mentioned these will primarily be outside the scope of this work.
1.3. Structure of the study
Chapter 2 goes through a survey of existing literature on the topics of TTCN-3, test parameters and ease of use issues in GUIs. The research problem is also explained in further detail. Chapter 3 explains the method of solution and the design I chose as the solution. In Chapter 4 the merits of the solution are discussed and problem areas
highlighted. Future research directions are also considered. Chapter 5 offers conclusions gained from this project and summarizes the results.