1. INTRODUCTION
1.7. ACTUAL RESEARCH WORK
Evaluation in Information Systems
In general, constructive research is driven with usefulness aspects. The utility of the artefact for its users should be evaluated. The environment establishes the requirements upon which the artefact is evaluated79. The evaluation is carried out to determine how well the artefact actually performs.
There are two cases in the evaluation: (1) the artefact is a totally new outcome, or (2) the artefact competes with old outcomes. For new outcomes, evaluations are not required. The outcome itself is regarded as the merit, but the potential importance of the construct could be evaluated. The originality of the solution should be demonstrated. If the construct has predecessors, i.e. the solution already exists in a certain form, March and Smith recommend evaluating the research in the following way:
“The significance of research that builds subsequent constructs, models, methods, and instantiations addressing the same task is judged based on “significant improvement”, e.g.
more comprehensive, better performance.”80
Järvinen81 recommends that even a new artefact be evaluated by considering the utility of the artefact. He suggests using an evaluation question to test the superiority of the new outcome, i.e. the researcher asks how the new artefact is better than the old ones at least in some sense.
1.7. ACTUAL RESEARCH WORK
My general approach in this study is very pragmatic. It is said that the colour of the cat does not matter, if it hunts mice. The saying describes my attitude towards my construct, the procurement software. There were hectic times, when I was maintaining the software in one project, generating the next software version, and trying to find the logic for new application features. Without using any profound scientific reasoning, I applied available working solutions to practical problems.
79 Hevner, March, Park and Ram: Design Science in Information Systems Research (2004) MIS Quarterly, Vol. 28, No. 1, p. 85
80 Marchand Smith: Design and Natural Science Research on Information Technology (1995) Decision Support Systems, Vol. 15, No. 4, p. 260
81 Järvinen: On Research Methods (2004) p. 109-110
Järvinen: On Research Methods (2004), p. 103 Goal State
Final State Initial State
Specification
Purchasing Realized Artefact
Implementation Specifications and Building in Parallel
Specified Goal
My research is based on the idea that it is possible to program procurement software following upgraded project instructions. The software would ease procurement in project environments and solve problems in (1) procurement, (2) project management, and (3) data management. Therefore, the research can be seen either as a thorough software engineering or as a knowledge management task. The software engineering tasks in this research are supported by the theories of information system life cycle, software development and software maintenance. Knowledge management was needed in gathering experiences from successful projects and sorting out the best practices in the projects. Based on the project experience, instructions, standards and sample documents were provided to support project implementation.
The procurement problems were supposed to be overcome by programming procurement software following standardised practices in procurement and project management. The software comprised all three inquiry round types, the necessary supplier and client correspondence documents, document flow management and reporting. Project management problems focused on the progress estimations of each purchase item and the accumulated over-all procurement status in projects. The progress calculations needed a better method than manual calculations based on the accumulated contract figures. Data warehousing features of the procurement software were seen as a part of knowledge management disseminating the existing knowledge within the company.
In the wide perspective of science, the research follows the design science approach. It seeks better solutions to the procurement function than the existing ones. It makes the research empirical, applied and value-laden. The research is empirical, because it concerns practical matters, seeking useful answers to real problems. It is applied research, because it applies theories of business science and Information Systems into practice. It is value-laden (or normative in Olkkonen’s terminology), because it aims at improvements; it seeks a more efficient way to carry out the procurement function in industrial plant projects. My study was not intended to follow the iterative spiral research model, but in practice it did; six development rounds were needed to achieve the final version of the procurement software.
1.7.1. RESEARCH MATERIAL
As discussed in Subsection 1.5.5., I consider my research as constructive research. I admit that there are case-study features in the beginning of the research. The knowledge management tasks for Pöyry Oyj, especially disseminating the gathered information, resemble a single-case study (one company, Pöyry Oyj). The other knowledge management task, rewriting of the project manuals, can be understood as a multiple-case study (many projects).
In principle, the analysis of problems before software development matches Yin’s idea of a case study82. He has defined the scope of a case study as an empirical inquiry that investigates a contemporary phenomenon in its real-life context when boundaries between the phenomenon and the context are not evident. He states that the case study method allows retaining the holistic and meaningful characteristics of real-life events, for example of individual life cycles, organisational and managerial processes, and neighbourhood change.
The research material was gathered from professionals of Pöyry Oyj. Not all projects were studied for the rewriting of the project instructions, only the most successful ones. It has to be remembered that the major mill reference list comprised nearly 300 projects already in 1991. The other factor influencing the selection of projects was the availability. If the key project professionals were
82 Yin: Case Study Research: design and methods (2003) pp. 2-16
available, their experience was used in formulating the project instructions. Later I extended the research to cover procurement status calculations as well. I developed a workload based progress follow-up, when I supervised the programming of the ERP modifications (I worked for ERP solution providers Tietonauha Oy, Minerva Suomi Oy,Tieto-Enator Oyj and Sentera Oyj, respectively).
Finally, I consider a project as a basic unit of analysis in my research, not a company. Even the software was required to manage several projects simultaneously. Therefore, I see that the number of projects rules out the single-case study option. I agree that my research can be considered to have been a multiple-case study before the programming of the procurement software started.
1.7.2. GOALS OF THE ACTUAL RESEARCH
In the beginning, the research concentrated on practical goals placed on developing the procurement function in large industrial plant projects. The practical goals were defined for: (1) the upgrading of the project manuals, and (2) the development of the procurement software. The improved project manuals were expected to meet current standards in project management, particularly in industrial plant projects. The procurement software was aimed to decrease the cost of the procurement function in project implementation and produce savings in the project implementation. The costs were supposed to reduce directly through the decreased workload (decreased labour cost) and indirectly via cost-efficient decisions in project implementation. Both the project instructions and the procurement software were expected to be used in all Pöyry companies.
The scientific contribution arises from the usefulness and the novelty of the project instructions and the procurement software. Design science research83 concerns through the building and evaluating of innovations against identified needs, and aims at the utility of the developed artefacts. Because the artefact has a purpose, it must be useful in its domain and it should solve a specified problem. A similarly crucial requirement for proper research is novelty. The artefact must be innovative, solving an unsolved problem or solving a problem better than any other solution before.
In business science, the constructive research approach84 means problem solving through the construction of models, diagrams, plans, organisations, etc. From the view of business science, the project instructions and the procurement software are the constructs. The project instructions were expected to guide the project implementation in future projects. The software was targeted to help in carrying out procurement tasks in large industrial plant projects and minimise the workload, particularly in procurement status estimation and data warehousing. The use of the software was assumed to produce savings in project implementation.
From the view of Information Systems, the situation has more subtleties. The expected results85 are grouped as constructs, models, methods and instantiations. In my research, the design (description) of the procurement software is a model, the programmed software is an instantiation (a realisation of an artefact in its environment) and the developed procurement practice, preferably applied with the procurement software, is a method.
I placed several success criteria to the research, as listed in Table 5 on the next page. It should be born in mind here that my role as an apprentice in the developing of project instructions differs radically of my role in software development. In software development, I was responsible for all the
83 Hevner, March, Park and Ram: Design Science in Information Systems Research (2004) MIS Quarterly, Vol. 28, No. 1, pp. 75-105
84 Kasanen, Lukka and Siitonen: The Constructive Approach in Management Accounting Research (1993) Journal of Management Accounting Research, Vol. 5, No. 3-4 (Fall 1993), p. 243
85 Marchand Smith: Design and Natural Science Research on Information Technology (1995) Decision Support Systems, Vol. 15, No. 4, p. 258
development stages. My success criteria can be seen to derive from the success criteria of design science86 (cf. Table 4 in Subsection 1.5.1.). My criteria follow the guidelines for design science, although I hesitated to include the requirement of the viable solution. I easily agree that it is the precondition for successful research that the solution works, but I just cannot see non-working software as an IT solution. For me, non-working software is an IT problem.
Table 5. Success Criteria of the Implemented Research
Design Science Guideline Project Instructions Procurement Software 1. Design as an artefact
x a viable construct Project instructions are rewritten. Software is programmed with adequate features and documented.
2. Problem relevance x solutions to relevant
business problems
Procurement and its follow-up is a vital part of any industrial plant project.
3. Design evaluation
Software is used in industrial plant projects (multiple-case study).
- gathering of company knowledge - structuring project instructions - business process redesign Evaluation:
- market tests
- financial impact of software
- requirements compared to software 6. Design as a search process
x utilising available means to taken into use. Based on user experiences, the software has been
- marketing material of project services
Documents and presentations:
- user interface manual - software manuals
- marketing material of the software
1.7.3. COURSE OF THE ACTUAL RESEARCH
My research consists of business process development and application development. These main research phases overlap, but the chronological order followed roughly this order. The general course of the actual research is illustrated in Figure 16 on the next page. The actual timing and order of the research tasks is discussed in the following subsection.
The business process development comprised knowledge management and company practice formulating tasks, which were carried out to revise project instructions. The knowledge management approach was used to formulate company practices in project management and procurement. These
86 Hevner, March, Park and Ram: Design Science in Information Systems Research (2004) MIS Quarterly, Vol. 28, No. 1, pp. 82-83
practices were described in project instructions. The project manuals were generally assumed to describe the state-of-art in project management and procurement in industrial plant projects.
After the rewriting of the project manuals, the development of the procurement software begun. The rewritten manuals were considered as application requirements, because they described the procurement function and the environment of the software. The software was programmed along project management ideas discovered and created in the company. The first two software revisions were programmed before the redesigning of the procurement processes using a linear model of procurement flow. The redesigning of the procurement software enabled more efficient solutions for procurement tasks by making the procurement flow circular.
Right after the introduction of the first procurement software version, the documentation of the software was organised. The created construct was described in user manuals: User Interface Guide87, General Programs88 and Purchasing Manager89. Internal training courses were held to familiarise project workers with the software. Finally, it was recognised that the software changed the procurement tasks and the impact of the software had to be taken into account in project instructions.
Figure 16. Actual Course of Research
Business Process Development
Pöyry Oyj decided to develop its business processes to meet international competition. The development of procurement processes included searching of best practices, standardisation of procurement and project management tasks, and organising knowledge management for the procurement function. The renewed project and procurement instructions, published as project manuals, were the significant results of the process.
The knowledge management approach gives a wide picture of managing knowledge within the company. It was applied in documenting the most recommendable project practices. The term
“knowledge management” was never used, because the term was not in common use at that time.
However, the principles of knowledge management were present and they were intuitively used. In retrospect, knowledge management was used in gathering experiences from successful projects and sorting out the most successful practices of these projects. Project instructions were written, standards were created, and sample documents were provided to support project management.
87 Vehviläinen, Pöyry Oyj: User Interface Guide (1990, re-edited in 2005), 32 p.
88 Vehviläinen, Pöyry Oyj: General Programs (1991, re-edited in 2005), 36 p.
89 Vehviläinen, Pöyry Oyj: Purchasing Manager (1991, re-edited in 2005), 55 p.
Need for Development
- international competition - business environment changes
Pressure for Development - personnel exchange - IT development Business Process Development
Knowledge Management Best Practices
Project Instructions
- project management instructions - procurement instructions
Procurement Application - procurement process - procurement follow-up
- document flow and data warehousing Application Development
Redesigning of Procurement Processes Procurement Software Programming
The best practices approach was used in restructuring the project instructions, which was started to improve the company’s competitiveness. Practices in several projects were studied and they were evaluated. Based on this benchmarking, the most promising work methods were used as basic material for the rewriting of the project instructions. The rewritten project instructions were regarded to comprise the most recommendable practices in project implementation.
Software Development
The idea of harnessing the knowledge with software arose after revising the project and procurement instructions. The software was targeted to support the procurement tasks. The software was expected to automate some business correspondence and reporting, and to enable fast, justified decision-making based on gained experience. The company goal was to decrease the needed working hours and professional skills in project implementation producing time and cost savings.
Finally, the software would force project professionals to follow the formulated best practices.
The software development can be divided to three, partly overlapping, major stages: (1) software development following directly the rewritten project instructions, (2) redesigning the procurement software, and (3) software development following the redesigned procurement flow. I was appointed to make the software development from design to programming and documenting. For testing I got some assistance from my colleagues in the procurement department at Pöyry Oyj. My supervisors motivated me to high-quality software development by arguing that I might end up using the software myself. In principle, it makes the software development as some kind of end-user computing.
The major stages consist of the development cycles of the procurement software, resulting in the corresponding software version. The first major stage produced software versions 1 and 2. The second major stage, redesigning of the software, took place during the third development cycle, which introduced software version 3. The third major stage covers software versions 4, 5 and 6. The first software version was able to carry out all the required purchasing tasks, but there were no interfaces to other systems. The second version introduced an interface to scheduling programs and data warehousing. The third and fifth application versions were programmed to utilise new features of the database engine. The fourth version brought document flow management to the procurement application. Finally, the sixth version solved the problem of procurement status estimation.
1.7.4. PROGRESS OF THE ACTUAL RESEARCH
I did not work at Pöyry Oyj in the 1970s, when Paul Talvio wrote the Pöyry project manuals. I respect his work and I consider his manuals as a preliminary step for my research. The revising of his project manuals started the actual research. I was appointed to the task as a secretary, mostly responsible for graphical illustrations in the manuals. I was supposed to acquire the necessary background information of procurement tasks in the revising of the project instructions.
Soon after revising the project manuals, I was appointed to program the procurement software following these manuals. The first and second software versions were programmed directly according to the procurement instructions. The versions were not very successful, because they did not satisfy the user needs well enough. It took time to recognise the problem. It was even more difficult to become convinced that the document-based models of the project instructions were not an ideal starting-point to application development. A shift of thinking was necessary; I replaced the document-based models with process models. The process models helped understand procurement and combine the processes of budget inquiries, tender inquiries and final inquiries. The process
model also helped in automating the follow-up of the procurement process. In retrospect, it is easy to understand that the document-based models of the manuals did not provide proper programming definitions. The time span of the research with Talvio’s work is shown in Figure 17.
Figure 17. Actual Progress of the Research
Actual Design Process
Simon90 describes design to be concerned with things that ought to be, with devising artefacts to attain goals. He sees the nature of the design process as a cycle, in which alternatives are generated and the generated alternatives are tested until a satisfying solution is found. He also brings up two levels of cost-effectiveness in the design process: (1) cost-effectiveness of design work, i.e. using resources sparingly in designing, and (2) designing economical artefacts with a high costbenefit -ratio. The required cost-effectiveness of the research explains some features of my research. I did not program competing procurement software versions and select the best version based on end-user satisfaction. This was impossible, because of the limited time, funding and human resources.
Instead of programming several alternatives, it is customary (at least in ERP business) that the design of the software continues until a working solution is found. The scarcity of time and money causes that any working solution satisfying the software requirements is acceptable as long as it is a better solution than the competing manual solutions. At that moment, all the other possibilities are abandoned. The selected idea is programmed (at least prototyped) and the program is introduced to the users. This first solution will be developed further, if there are weaknesses or improvement possibilities in the solution. On the basis of the first version, the users propose improvements. The improvements are programmed and the next software version is presented to the users and so on, until satisfactory software is implemented. After each selection, the other possibilities are neglected and they are not documented, just like design and programming errors are not documented. This
Instead of programming several alternatives, it is customary (at least in ERP business) that the design of the software continues until a working solution is found. The scarcity of time and money causes that any working solution satisfying the software requirements is acceptable as long as it is a better solution than the competing manual solutions. At that moment, all the other possibilities are abandoned. The selected idea is programmed (at least prototyped) and the program is introduced to the users. This first solution will be developed further, if there are weaknesses or improvement possibilities in the solution. On the basis of the first version, the users propose improvements. The improvements are programmed and the next software version is presented to the users and so on, until satisfactory software is implemented. After each selection, the other possibilities are neglected and they are not documented, just like design and programming errors are not documented. This