6 RESEARCH METHODOLOGY
6.4 Framework
The main goal of this study was to create a framework to guide organizations with the implementation of a BI system. The idea of the framework is to contain a comprehensive package of information but to keep each topic of it compact.
As mentioned in the previous subsection, the framework will be evaluated, tested, and improved in a continuous cycle.
The case organization of the study is operating in the public sector. How-ever, a claim could be made that in this study, the sector of the case organiza-tion does not limit the framework that much. As it was found in the second sec-tion, there are many differences between the two sectors, but they do not neces-sarily affect the architecture and technologies of the BI system. The major dif-ferences were that the public sector was found to be more transparent and less competitive (Nutt, 2005). Additionally, the public sector is considered to be less flexible and it doesn’t have as clear goals as the private sector organizations usually have. These definitely are factors that should be considered in the framework, but at the same time, they are not limiting it too much. As said, the framework could be marginally more useful for public sector organizations, but it is still usable for the private sector as well. However, this consideration will be inspected more closely further in the study.
6.4.1 Development
The construction of the framework will begin with the first step of the DSRM process model, which was identifying of the problem and motivation. The iden-tifying will be done to the problem of the case organization and it will be
pre-FIGURE 7 DSRM process model (Peffers et al., 2007)
sented as a research question. In practice, this means that the problem must be first identified, then defined, and then it is possible to understand what main objectives of the study are. Roughly, the main problem of the case organization was known in the very early stage of the study, because the right direction was needed to build an appropriate theory base. Conveniently, the next step of the process model is to define the objectives. This step requires knowledge of the current problems, current solutions, and the efficacy of the solutions (Peffers et al., 2007).
After the problems and the objectives are defined, it is possible to move on to the third step, which is the design and development of a solution. The artifact, which in this case will be the framework, will be first constructed based on the theoretical knowledge base of the study. When the first version is ready, it can be tested and evaluated.
6.4.2 Demonstration and evaluation
Without any evaluation, the designed artifact will only act as a hypothesis that it could possibly solve some kind of a problem (Venable, Pries-Heje & Basker-ville, 2012). Even though the demonstration and the evaluation are two differ-ent phases with differdiffer-ent purposes, both are conducted by using similar logic to find out if the artifact fulfils the purpose of it. Venable and others (2012) have found five different purposes for evaluation in design science research. The purposes are the following.
1. Evaluate the artifact for its utility and how well it achieves its purpose 2. Evaluate the formalized knowledge and design methodologies that
were used to build the artifact
3. Evaluate the artifact by comparing it to similar artifacts
4. Evaluate the artifact by the possible negative or positive side effects of it
5. Evaluate the artifact to identify its weaknesses or areas of improve-ment
Fortunately for the quality of the study, there is an opportunity to demon-strate the utility in the case organization, which works as a very viable testing ground. There are various possible ways to test a theory or framework (Peffers et al., 2007), but considering the research topic, its scope, and the project itself, there are not many viable methods. Few examples of the problems will be pre-sented. If the quality of the framework would be evaluated by the quality of the system that has been built with the help of the framework, there would be few problems. Firstly, the quality of the BI systems can be affected by unfortunate conditions which have nothing to do with the framework itself, for example, a lack of resources. Secondly, if the performance of the final system would be evaluated, it would prolong the study by a great time. Considering the measure of the project and the nature of the context environment, it takes a long time for
the project to be complete. The evaluation method of this study will be follow-ing the first and the fifth purpose of Venable and others (2012). Naturally, in an optimal situation, all the five evaluation purposes would be used, but it is not likely that all of them would be required for justifying the artifact. The objective or purpose of the framework is to provide guidance for organizations and to be comprehensive enough to cover the most important areas of the research topic.
By evaluating how many issues of the case organization the framework can solve or help with, could be considered as a sufficient evaluation method to draw fairly solid conclusions. To avoid evaluating the quantity of information over quality, the framework has to truly offer help with considerations, so that exact topic can be accepted as being covered or sufficient enough in the frame-work.
Hevner and others (2004) have categorized various evaluation methods for different types of DS studies (Table 5). There are 5 different evaluation types, which each contain a few different evaluation methods. This study fits best with the observational category because the case study is the closest description of how the study will be conducted. However, it could be argued that the evalua-tion method is not purely only a case study. For example, the method that will be used for this study also meets the description of the dynamic analysis.
The methodology of testing will be following the fourth and fifth steps of the DSRM process model, which are demonstration and evaluation. The demonstration step means that the artifact will be used for example in a simula-tion or in a case study, and in the evaluasimula-tion step, the fulfilment of the objec-tives defined in the second step is studied (Peffers et al., 2007). Naturally in this study, the demonstration step will be conducted in the case organization. Con-sidering the possible ways to evaluate an artifact, the previously listed context-specific problematics, and the nature of the implementation project, the evalua-tion of the framework will be made based on how comprehensive and helpful it is.
As mentioned earlier, the first version of the framework will be construct-ed by utilizing only the previous theoretical knowlconstruct-edge. However, if any short-ages or flaws are found in the demonstration and evaluation phase, changes will be made. In this case, the demonstration and evaluation phase will be re-peated until the framework satisfies the problems of the business environment.