KONE Corporation is one of the world’s leading manufacturers of elevators, escalators and building doors. It has made several important innovations during its existence to for example elevator technology, like MonoSpace, an elevator without machine room, in 1996 and UltraRope, carbon fiber rope for elevators, in 2013. Research and development (R&D) plays a very important role at KONE, and the number of people working in R&D at KONE’s Hyvinkää office has grown significantly during the last 10 years. KONE has also made its product quality as one of its top priorities.
There is software in every machine KONE produces, and the increasingly sophisticated needs of people put pressure on its development. Software development is a major area of interest in R&D at KONE as there have emerged new ways of using the machines they produce, new needs of customers and thus also a need to invest in software. There is a need to constantly develop the processes of the software development, to make sure that software development is effective and to minimize the number of software defects both during the internal development process and especially at the customer surface.
KONE has used automated software testing in its software development since the early 2000s, starting with safety related software of elevators and broadening its use consistently within the company’s software development. There are different ways of making automated testing, and during this thesis there are three different software development projects of KONE under study. An automated testing team, which serves these three projects among other projects, was established in 2012.
There are differences of test automation use between these projects, as two of them have started using its use right from the start of the project and one of them has started its use in the midst of the project.
Test automation’s profitability is a somewhat hot topic in software development, and the Internet is filled with return on investment (ROI) calculators of test automation. These calculations often handle testing as a set of tests, and the profitability in these calculations is seen as time savings if the amount of time used to make the automated tests and the execution of them becomes smaller than the same thing done with manual testing. This point, however important it is, covers only one element of test automation’s profitability. Especially in so called agile software development methods testing should be a continuous process and when done properly, automated testing plays in a big role in the whole development process. Test automation often requires a significant amount of continuous work in a large industrial setting of embedded software development, which is why there are many enablers and disables of its use which simple calculators don’t take into account. All tests cannot, however, be automated, and the time savings are just a part of the benefits of test automation.
There are many studies done about test automation and its profitability, but even they often cover only some aspects of it and there are few studies that address all of the effects it may bring to a software developing company. The reason for that is perhaps that there are many ways to implement test automation to software development and project context must be taken into account when trying to address whether the investment in it is worthwhile. A lot of software projects fail and software defects can be costly not only for the company but for the whole society too. A research done in 2002 calculated that software defects caused yearly costs of 59,5 billion dollars for the US economy, and this was largely due to inadequate resources that companies use to their software testing (Tassey, 2002).
Test automation can thus be seen as an investment to software developing process, and it is a kind of investment and process that usually has a long lifecycle in a company. There is thus also a need to justify its existence financially, because the ultimate aim of a company is to make profit to its owners. A comprehensive cost-benefit analysis of test automation has previously not been made at KONE
regarding test automation with the setting under study during this thesis work, and because it has been a part of their software development for only a short time, there hasn’t been much data to analyze its effects to both software and product development in general.
According to some sources, testing and rework can take even half of the total time developing new software. Thus, there is a clear need to shorten and optimize the time to test the software. Computers can execute a test, which manually could take minutes, in just a few seconds or less. It also enables testing at night or other non-office hours and long simulations of the software at work. Automated testing can therefore shorten the time to test the software and it is especially good at regression testing, i.e. when a test must be executed multiple times due to changes in the software to ensure that previous functionality still works. When the tests are being executed immediately after changes to software code is made, test automation systems gives direct feedback to the developers of the software. Test automation also enables testing software with a great number of input data, which would be impossible to generate manually.
Test automation, along with alternated ways of work can be seen as a way of software process improvement (SPI). Earlier fault detection and acceleration of different testing phases could lead to an acceleration of a whole software development process. The later the defects are noticed, the more it takes time, and therefore money, to fix them. It is harder to trace the reason for the defect, and possibly other parts of the code have to be changed and tested again. The late findings of defects can lead to delays of product releases and, if they are found after the product has been launched, to repairing costs and a possible loss of reputation.
Therefore test automation can, in addition to process improvement, be seen as an investment to software quality improvement. The more there are tests and the more times they are executed, the more thoroughly the software is tested and the more defects are found. Therefore is could be concluded, that more testing
generally means better quality, i.e. less defects in the products when they are already in use, which means fewer updates of the products already in use, which could be costly to the company.