Creating a maturity model for a document management system

(1)

Klaus Simolin

CREATING A MATURITY MODEL FOR A DOCUMENT MANAGEMENT SYSTEM

Master of Science Thesis Faculty of Management and

Business

Samuli Pekkola

Henri Pirkkalainen

June 2021

(2)

ABSTRACT

Klaus Simolin: Creating a maturity model for a document management system Master of Science Thesis

Tampere University

Master’s Degree Programme in Information and Knowledge Management June 2021

The purpose of this study was to find how maturity models could be used to evaluate the current state of a document management system, in use in order to better understand customer’s situation and ways to improve their satisfaction. The study consisted of three theoretical fields:

document management systems, system life cycle and total cost of ownership and maturity model. The focus was to introduce these subject fields, so empirical portion could be built on it.

As the number of documents at companies increase, the need of document management systems has increased. Document management systems (DMS). The main goals of a document management systems are to increase the findability of documents through the use of properties or metadata, provide a framework to enhance document life cycles for better transparency and provide actionable and valid data on documents for decision making. To make the best DMS possible, the system life cycle and costs have to be considered. Life cycle can be categorized into initiation, system development, deployment, usage, further development and shutdown. The total cost of ownership consists of all the cost accumulated during the life cycle. Maturity model depict the current state that the system in use is. The model consists of maturity levels and chosen process areas.

Based on the theoretical and the twelve interviews held, a maturity model for document management systems was created. The model is based on the Capability Maturity Model Integrated (CMMI). It consists of the process areas users, system usage, maintenance and improvement, vault structure and cooperation model. The maturity five maturity levels chosen were not identified, identified, defined, managed and optimization.

Keywords: Document management, DMS, Maturity model

The originality of this thesis has been checked using the Turnitin OriginalityCheck service.

(3)

TIIVISTELMÄ

Klaus Simolin: Kypsyysmallin luominen dokumenttien hallintajärjestelmälle Diplomityö

Tampereen yliopisto

Tietojohtamisen koulutusohjelma Kesäkuu 2021

Tämän tutkimuksen tarkoituksena on selvittää, kuinka kypsyysmalleja voitaisiin hyödyntää asiakkaan tuotantokäytössä olevien dokumenttien hallintajärjestelmien nykytilan arviointiin asiakastyytyväisyyden parantamiseksi. Tutkimus koostuu kolmesta teoreettisesta aihealueesta:

dokumenttien hallintajärjestelmät, järjestelmän elinkaari sekä omistuskustannukset ja kypsyysmalli. Tavoite on esitellä nämä aihealueet, jotta empiirinen osuus voidaan pohjustaa niihin.

Asiakirjojen määrien kasvaessa yrityksissä dokumenttien hallintajärjestelmien tarve on kasvanut. Dokumenttien hallintajärjestelmien päätavoitteet ovat lisätä asiakirjojen löydettävyyttä metatietojen avulla, tarjota puitteet asiakirjojen elinkaaren läpinäkyvyyden parantamiseksi ja luoda dataa asiakirjoista päätöksentekoa varten. Parhaan mahdollisen dokumenttien hallintajärjestelmän tuottamiseksi järjestelmän elinkaarta ja kustannuksia on harkittava. Elinkaari voidaan luokitella osa-alueisiin aloitus, järjestelmän kehitys, käyttöönotto, käyttö, jatkokehitys ja lopetus. Omistuksen kokonaiskustannukset koostuvat kaikista elinkaaren aikana kertyneistä kustannuksista. Kypsyysmalli kuvaa käytössä olevan järjestelmän nykytilaa. Malli koostuu kypsyysasteista ja valituista prosessialueista.

Teoreettisen osuuden ja kahdentoista asiantuntijahaastattelun perusteella luodaan kypsyysmalli dokumenttien hallintajärjestelmille. Malli perustuu CMMI (Capability Maturity Model Integrated) -malliin. Prosessialueiksi valittiin käyttäjät, järjestelmän käyttö, ylläpito sekä kehitys, varaston rakenne ja yhteistyömallista. Viideksi kypsyysasteeksi valittiin ei tunnistettu, tunnistettu, määritelty, hallittu ja optimoitu.

Avainsanat: Dokumenttien hallinta, DMS, kypsyysmalli

Tämän julkaisun alkuperäisyys on tarkastettu Turnitin OriginalityCheck –ohjelmalla.

(4)

1. INTRODUCTION

1.1 Background for the research

The amount of information used and stored has been exponentially growing, which has caused companies to put more emphasis into managing information more efficiently. As a solution to managing information, companies and public sectors are in the process of minimizing the use of printed documents and replacing traditional documents with electronic documents. Due to this transformation, the volume of managed electronic documents has begun to climb rapidly, thus resulting in the need for competent document management systems (DMS) and processes capable of handling millions of electronic documents.

The target company M-Files is a document management system that aims to tackle the problems with finding documents, version control and document processes by classifying documents using metadata that is specific to each customer. The customizability of the system causes each implementation case to be unique in its own way, and it cannot be used with another customer in an as-is basis. The end result of a long process is that there are thousands of solutions built to customers with the same core but with different customer specific customizations.

The foundation of a successful system improvement, after its implementation, is not only based on the desired state or goal of the system, but in the understanding of the current state of the system. As M-Files systems are configured differently to each customer, understanding the current state of one system has been difficult. The goal of this thesis is to create a tool for the M-Files consultants to use in determining the current state of a customer’s system, so they can better plan for its development.

1.2 Goals and the research questions

The goal of the research is to create a maturity model for a document management system. The thesis uses scientific writings to analyze document management systems and maturity models in order to find ways in which maturity model can be used to model a document management system. The goal is to find central factors that contribute to the current state, and a way to evaluate these factors. The empirical portion of this study

(7)

utilizes the theory to produce a concrete maturity model tool for M-Files. The study strives to give tangible results that will help the work of consultants at M-Files.

The research problem of the study is the utilization of a maturity model for a document management system. The actual way to measure different areas of maturity and specific maturity levels of each area are ruled out of scope of this research.

From the research problem we derive the main research question:

“What kind of a maturity model can be used to analyze the current state of a document management system in production use?”

The question can be divided into sub questions, which help to answer the main research question. The sub research questions are:

 What are the central features and challenges of a document management system?

 How should different areas in the maturity model be appraised to find their level?

 What maturity model will provide the greatest benefits for analysis?

 What areas and levels in the maturity model should be used in a document management system?

1.3 Focus and research boundaries

The research handles two different themes: document management systems and maturity models.

Research areas

(8)

Thesis boundaries have been made to study only cases dealing with software development. Software development on turn is limited only to its improvement and maintenance phase of the life cycle. This is further limited to systems developed to customers, and any internal system development is left out of the scope of this thesis. Due to these limitations, organizations that are in the defined scope act in the B2B-business (business-to-business), so the customers are not the end-users but their companies.

Similarly, only maturity models tied to software are studied and models tied to other processes or systems are left out of the scope. The scope only includes the creation of maturity models such as defining areas and levels, thus leaving out the implementation and maintenance phases of the maturity model. Furthermore, the actual methodology of measuring the areas is not included in the scope, for example, the format in which the documentation of a process has to be in order to merit a certain level in a defined area.

The scope of the theory for document management systems is centered on DMS’s features and typical challenges after longer period of usage. Scope does not include DMS’s life cycle, nor does it include evaluation of design choices or solutions to the typical challenges presented in the portion. Overall, the theory aims to provide a comprehensive overview of document management systems and their typical problems in their pro- longed use and maintenance.

1.4 Methods

In order to open up the research problem, formal concept analysis was chosen as the method for the theory portion. The research problem is dismantled it into smaller entities, or sub research questions, that will be researched separately. The information ascer- tained from these is then combined together to form the complete analysis. The purpose of formal concept analysis is to construct logical conceptual structures that can be combined to form rational human communications (Wille 2005, p. 2). The concept analysis is a theoretical representation of the topics introduced by the research problem. Repre- sentation has been done by reviewing scientific writings on the chosen fields. Sources include all the chosen topics of the study: document management systems, maturity models and system improvement and maintenance.

The empirical portion of the research was done by semi-structured interviews. The first main questions were the same for everyone, but toward the end of the interview a topic

(9)

or two were chosen and the final questions were focused on them. This was done because interviewees were from different fields of expertise, and this was taken into con- sideration in the latter portion of the interview.

1.5 Thesis structure

The Thesis consists of an introduction in addition to the theory and empirical portions.

The theory portion is divided into four different chapters. Each chapter starts of by defining the central concepts and then moves into more in-depth topics toward the end of the chapter.

Thesis structure

The empirical portion consisted of the interviews of twelve subject material experts of M- Files. Based on the information collected from the theory portion, a maturity model is chosen as a base and initial maturity levels are defined. The data gathered from the interviews is used to define the different areas for the maturity model. Using all of this, the maturity model is created.

(10)

2. DOCUMENT MANAGEMENT SYSTEMS

As the amount of information keeps growing, companies and public sector organizations are forced to move away from printed documents towards more manageable digital documents (National Digital Library 2011). The electronic documents are stored in a document management system (DMS), which can be a collection of network folders or a more complex application that is built for the sole purpose of document management.

Swanson and Culnan (1978) observed the need of document-based computer systems for management’s preparation and control. According to their study, information found in documents has widely been comparatively ignored, and they suggest that document- based systems will eventually start to be used in companies. (Swanson & Culnan 1978) Similarly a decade later MCLeod and Jones reported (1987) that the decision making of CEOs was much more highly based on written memos, phone conversations and non- computerized reports than before (McLeod & Jones 1987). In the present day, companies manage documents daily in order to create and store information causing pressure for more advanced document management systems and processes.

Document management can be handled using an application system designed for document management or it can be done without an application. According to Anttila (2001), document management without an application is often based on folder structure and file naming making the search for specific documents slow and more difficult as the number of users increase. This kind of document management does not offer tools for concurrent modifications and advanced version control. (Anttila 2001) For the purpose of this thesis, only document management systems that use an application are examined because that is the most efficient way to handle large amounts of documents and users. Henceforth, references to document management system refer to document management system using an application.

2.1 What is document management

Document management is a procedure of storing specific documents for a certain use or reason (Salminen 2000). A document is any collection of data that is meant to be human readable as a source of information. Based on their internal and external properties, documents can be classified into different document types, which are, for example,

(11)

memos, reports, minutes, guides, manuals, drawings and diagrams. Documents are created at an ever-increasing base, thus creating pressure for document management.

(Anttila 2001)

2.1.1 Document life cycle

Another important aspect in document management is the document’s flow or knowledge creation process. Document management is closely tied with document life cycle, which consists of all the document’s phases from design to its destruction. (Ralph 1995) The flowchart below (Figure 3) shows a simplified way to represent the workflow. Document management should take into account documents in all of its life cycle periods, so that information is accessible and ever evolving.

Document life cycle (Anttile 2001; Shanks 2005)

The figure above (Figure 3) is meant to give general principles for document life cycle, but the reality can be more complex or simplified. According to Salminen (2000), the design phase consists of planning the document’s structure, creation procedure and presentation. Choosing the technology for the document is done in the design phase because it can affect the rest of the document’s life cycle. (Salminen 2000) It is important to note that not all the documents begin their life cycle inside the company. External documents are documents that were created by an outside party but may follow the normal inspection and publishing procedures of company generated documents (Shanks 2005).

In the design and creation phases, many documents are created and some of them are destroyed before the publishing phase. Other documents are maintained as working documents that are modified constantly as more information is stored in them. (Shanks 2005) These working documents pose challenges, as the newest copy has to be found and simultaneous modifications have to be handled. User access control is vital part of document management to ensure the usability and integrity of the stored documents (Salminen 2000).

(12)

Some documents are modified even after their initial publication. These types of documents add a maintenance phase to the process. (Shanks 2005) A document might stay in a maintenance phase indefinitely as it is constantly revised with new versions. At the end of the document’s life cycle, it is destroyed or archived and destroyed after a defined time period.

2.1.2 Document classification using metadata

As the number of documents in the DMS grows, more profound ways of finding the right documents have to be found because users cannot go through all the documents in the system. Metadata can be used to classify documents to enable better searches. Accord- ing to Gill (2008) metadata is information about information and is often related to the card catalogue found in libraries. (Gill et. al. 2008) Metadata describes the information and without it managing the information is impossible. Some of the metadata is created automatically when creating documents, for example created by and date created, but the rest must be added by the user. In the case of document management, the purpose of metadata is to describe manageable documents. (Salminen 2005)

There are many classifications of metadata available, but the one often sited is the classification given by NISO in 2001. NISO (2001) categorizes metadata into three separate groups administrative, structured and descriptive:

1. Administrative metadata is passive metadata that is created automatically, for example, like the afore mentioned created date and created by. This data serves to help with property rights and that data is not lost.

2. Structural metadata is information of the container of the metadata. This could, for example, be the file format or size.

3. Descriptive metadata is active metadata inserted by the user, which user uses to classify the data to help finding it in the future. (NISO 2001)

According to Duval et. al. (2002), a few of reasons exist that give metadata classification an edge. Metadata is modular, which means that the metadata schema consists of mod- ules or parts that can be used to classify information. This provides a more flexible and maintainable storage basis for information. Secondly, metadata gives extensibility that is closely tied to modularity. Extensibility means that some of the metadata can be used to classify other information, but not all the metadata have to be extended to classify other metadata. The level of detail of the metadata can be altered, and this is called refinement.

A more detailed metadata will make the data more accessible but will create a higher burden on those who have to insert the descriptive metadata. Lastly, much of the

(13)

metadata can be considered to be multilingual, or otherwise very easy to translate. This creates great value for storages with users with different languages. (Duval et. al 2002)

2.1.3 Document processes and workflows

In order for an organization to function efficiently, common processes have to be defined and maintained. According to Davenport (2000), a process is a definitive chain of tasks with a beginning and an end, where individual tasks can be completed by different responsible people. In short, it is a structure for doing a certain thing within an organization.

(Davenport 2000)

The purpose of process modeling is to describe an existing process within an organization and make it formal by presenting it in an explicit format. Modeling can help to identify factors that weaken the process’s efficiency or to find concrete problems within the process like insufficient resourcing or organizing (Martinsuo & Blomqvist 2010). When modeling a process, it is not enough to arrive at the outcome, but it is important to describe each step needed to reach the outcome. The tasks needed to arrive to the given outcome can branch out and include repetition. Repetition happens until a certain condition is met.

(Kettunen & Simons 2001)

In order to model a process, often a workflow is needed to clarify it. A workflow describes the different states within a process and the stakeholders responsible for them. Accord- ing to Adams (2008), a workflow is a tangible approach to handle the requirements of a process. It provides the ability to create reports of workflow processes, thus providing data for statistical analysis within the organization. In document management, a workflow provides a seamless way for the document to move between people and transparency into what state the document is at any given time. (Adams 2008) Workflows add depth to handling documents, as it is possible for users to browse documents based on state.

For example, if the user oversees reviewing contracts, he can find all the documents waiting for approval (see figure 4).

(14)

Example of contract workflow

Figure 4 gives an example of a simple contract workflow. The purpose of the workflow is to depict the actual life cycle of the document. The different state names should be clear, so that the user can intuitively understand the current state the document is in. As the user makes decisions, for example in the state “2. Waiting for approval”, the options should be unambiguous. Automatic transitions should be used whenever possible to pro- gress the life cycle of the document. In the example, the document moves automatically to “3.1 Expiring” thirty days before expiration date and then again to “4. Expired” as the expiration date is reached. Users have always the ability to start the life cycle again if the contract is renewed. All the changes in the life cycle of the document can be browsed in the document’s history.

2.2 Document management using a system

Document management system (DMS) is a system used to store and organize documents in order to ease their listing and searching. In the system, the documents are classified via numerous properties that enable filtering them in different ways. (Anttila 2001) The more properties or metadata linked to the document, the more various ways the document can be displayed in the system. The system can also help in filling the metadata. Basic metadata such as file properties and creation dates can be easily auto- mated, but more advanced systems can use intelligence to discover metadata from the actual content of the document. According to Anttila (2001), a document management system consists of a database where data is stored and a management system. The management system has a user interface that enables document classification using

(15)

properties, search, forming views or folder structures, management of document permissions and version control (Anttila 2001). The figure below (Figure 5) shows an example of how metadata is displayed in the document management system M-Files.

Example of a metadata card in M-files

In the example above, the order of the metadata goes from highest importance to lowest importance. The class of the document answer to the question “What is this document?”.

The purpose of the class property is to give the first layer of classification to narrow down searches later on. Other metadata, such as customer and project, are used to link the document to different registries. For example in the above, the document is now found under the customer “A&A Consulting” as well as under the project “A&A DMS”.

The main functions and properties of a DMS include (Olehovych 2014; Raynes 2002):

 Loading and storing information (documents): In addition to the actual files, information tied to the files are stored in the system as well. This includes file specific metadata, but also more complex registries such as customer and project data, which can be linked to documents to ease their searching and displaying.

The stored information should be accessible and different versions of information visible.

 Document preview: The system should have a way for the user to preview the documents without actually opening them in a separate software. An example of this would be the built-in preview side panel found in Windows file explorer. The

(16)

preview can be used to find the correct document without having to open multiple documents. In some cases, the needed information can already be found using the preview and opening the document is not needed. This also enables the possibility to comment the document without having to edit the document itself.

 Archiving methods for old files and documents: As the document manage- ment system ages, the number of documents grows, which in turn makes it hard to locate a specific document in the system. Instead of destroying old files, it is better to archive them, so they can be found later in case they are needed later.

The system should include a way to automatically archive old and unused files and documents. The challenge is to identify which files are unessential enough to be archived.

 Indexing subsystem: An indexing subsystem is needed to allow fast search of information from the system. The subsystem regularly reforms the index to provide optimal search speeds. New information and documents are automatically added to the index by the subsystem.

 Search tool: The search tool is a very viable entry point into finding the correct document from the system. The search should include the possibility of an open search field that can be used to find text from inside the documents and any metadata tied to the document. In addition to this, a more advanced search should provide the possibility for the user to narrow the search to only include documents that have some specific metadata. For example, a search that only includes documents that have some specific project in their metadata.

 Document organization into views: Views are a powerful and a familiar entry point into finding documents from the system. Users are used to finding information from Windows folders, so views provide a similar platform where documents are organized into views using specific metadata they have. Examples of these kind of views are documents by customer and documents by projects. Us- ing these views, the user is able to find all the document pertaining to the chosen metadata. Users have individual preferences on how they search for information and how they like to view it, so providing different ways to find documents in- creases the usability of the system.

 Workflow for document life cycle: As most controlled documents have a very defined life cycle, it is important for the document management system to be able to provide a framework where to handle these documents. The workflow has to

(17)

be able to handle circulation and approvals. Most modern document management systems have a possibility to use electronic signatures, which further en- hances the process as paper versions of the documents are no longer needed.

As the document reaches a certain point in its life cycle that requires an action from a user, the user should be alerted automatically and/or the document should be raised in the user’s UI.

 Permission and privilege control: As the system contains a wide variety of the company’s documents, different kinds of access controls have to be implemented, so that only the right people see information deemed sensitive. The privileges can be controlled through metadata or through users’ access level, but the more they can be set automatically the better the usage for the end users. The permissions set for a document can automatically change as the document moves through its document life cycle in a workflow.

 Version control: Version control ensures that the latest version of the document is available, and that it is not modified by multiple users at the same time. Typi- cally, this is handled by check-out and check-in procedures. More sophisticated systems might include automatic merges of two branches of the same document.

Only one instance of each document is found in the system, and its history can be browsed. A previous version of the document can be rolled back as the current version of the document.

 Conversion of paper document to digital: Much of the older documentation is organizations is still found in paper format. The document management system should have procedures to convert these documents into digital format. These include automatic links from scanners, so that documents scanned are automatically added to the system. PDF formats should be made searchable, so they can be more easily found later.

Document publishing: Document publishing is the way the document can be viewed from the document management system. These extend beyond the normal ways of view- ing the document via a client. In more comprehensive systems, the document is also made available via methods such as mobile clients, document portals and public and/or private web URLs. Publishing can be a part of the document life cycle. As the document’s state reaching a state in which it should be made public, it is automatically flagged or transferred to a state where it is available, for example in a document portal.

(18)

3. SYSTEM’S LIFE CYCLE AND COST OF OWNER- SHIP

In order to understand the current state of a system, a wider understanding of the under- lying process is needed. Chapter 2 of this thesis defines what document management is, and what does it mean to manage it with a system. This chapter takes a closer look at the total life cycle of information management systems and costs pertaining to the maintenance of them. All of this impact the final value perceived by the company using the said system.

When evaluating a system’s current state, the whole process needs to be examined.

Decisions and mistakes done in previous phases reflect upon the current state. This chapter outlines the general life cycle of a system starting from initial project to usage and maintenance. To better understand any contingencies the system usage has, usage costs are also covered. The costs can be described by the “Total cost of ownership”

(TCO), which divides the costs into categories: acquisition costs, control costs and operations costs (David et al. 2002).

3.1 System’s life cycle

The purpose of defining the system’s life cycle is to create the basis for studying the costs and usage of the system. The purpose is not to study different types of life cycle models, but to give basis to other parts of the study. The model used in this study is based heavily on a model presented by Alter (2002). The phases chosen for this model are initiation, system development, deployment, maintenance and further development or system shutdown. (Alter 2001)

3.1.1 Initiation

System life cycle starts with the need for a new system. So first, it is necessary to define if there is a need for a new system, or if the existing system is sufficient for the needs of the organization. The question often is renewing and modifying an existing system or replacing it with another. (Alter 2001) Examples of different reasons behind deciding to replace an existing system are system’s age, high maintenance costs, difficulty adapting to IT-strategy, termination of support and IT-trends (Lyytinen & Newman 2008).

From the raised business needs, a preliminary report and system definition are created.

Their purpose is to describe main goals and boundaries set for the coming system.

(19)

(Ruuska 2001) With the use of these documents, a system provider is searched for, and it can be either inside the organization or an outside party. Much of the resources should be invested into finding the best system provider because finding the correct system provider is challenging. (Kaskela 2005) Finally, the delivery project is initiated where the customer and the system provider agree on practical ways of working and commit to the project on both sides (Tonnquist 2008).

3.1.2 System development

System development begins with making the requirement definition which in turn are changed into system functional definitions (IEEE, 1998). Using this document as a basis, a technical specification document is created containing the more technical features of the system (Haikala & Märijärvi 2006). The actual implementation is made based on this technical specification document. This process allows each feature of the system to be derived up to its customer requirement (Leino 2001). The system also includes system tests that includes all system functionality and performance measurements (Pan 1999).

It should also be noted that testing can be carried out in all parts of the system’s life cycle, but its role decreases towards the end of life cycle (Haikala & Mäkijärvi 2006).

3.1.3 Deployment

Although earlier phases of the life cycle would have been well implemented, deployment should in any case be a multidimensional process that demonstrates the ability to make the system a real business function (Markus et al., 2000; Lyytinen & Newman 2008).

Many things are complicated by the fact that it should take into account both technical and organizational details. (Lyytinen & Newton 2008).

Deployment begins with system installation, and its complexity depends largely on the needs of customized settings at various organizational sites (Markus et al., 2000). From a technical standpoint, the implementation can be implemented in different ways. The transition can be implemented in parts (migration), in its entirety (roll-outs) or pilots (Alter 2001). All options have their own risks that cause their own costs when the risk is realized.

In order for the implementation to be successful, the organization requires adaption to the new system. According to the Leavitt's model, the entire organizational activity can be divided into four distinct areas: people, technologies, structures and tasks (Keen 1981). Changes in one part of the sub-area are also reflected in other areas, so a comprehensive adaptation should take place in each of the regions.

(20)

The last step is acceptance of the system in the organization. This means that individuals agree to use the new system and change their own ways of working, so that they support the system and can be supported by the system. This can be described by the technology acceptance model (TAM) model described in Figure 6 (Legris et al., 2003).

Technology acceptance model (Adapted from source: Legris et al. 2003) Implementation should therefore take into account the needs of the end users so that the system will have actual use. According to the model, the system must be easy to use, and its functions and needs should be justified to the end user. In this case, the system is used out of end users’ own will and need, and not from an external pressure.

(Legris et al., 2003; Kim & Kankanhalli 2009)

3.1.4 Usage

The actual use of the information system in the lifecycle takes place when the deployment has been successfully completed. At first, full benefits from the system are not received. It is important to train people in the implementation phase, which can be considered part of the system's ongoing development (see 3.1.5). (Kettunen & Simons 2001) As part of the usage, maintenance of the system, which is a critical part of the life cycle, should be implemented. More time is used for it than for the system development.

Maintenance costs can be relatively high, especially if errors during development need to be corrected. (Swanson & Dans 2000, 277) An essential part of the usage is the continuous evaluation of the system. On this basis, it is possible to decide whether the purchase of the system was justified. Evaluation also makes it possible to review errors.

(Kankaanpää 2011) In addition, the evaluation provides knowledge for future projects.

(21)

3.1.5 Further development or system shutdown

The individual lifecycle of the system ends either in its further development or system shutdown. It is difficult to determine exactly where the system will move to further development. The basic assumption is that this is a larger-scale development project that con- siders the business aspects of the organization (Kettunen & Simons 2001). System shutdown, in turn, simply involves the decommissioning of the system and the resulting direct actions. According to Kim & Kankaanhalli (2009), system shutdown includes at least the following measures:

 Data transfer to a replacing system

 Archiving

 Removal of the hardware

 Communication to concerned parties (Kim & Kankanhalli 2009)

3.2 Total cost of ownership (TCO)

The life cycle defined in section 3.1 is the basis for the study on the total cost of ownership of the system. It is essential to look at costs in each life cycle phase to create a realistic overall picture. For the purpose of this study, the definition of TCO given by David'in et al. (2002) is reduced into two categories: acquisition cost and administration costs. Combining this with the specified life cycle, the different life cycle segments can be split as follows:

 Acquisition costs: initiation, system development and deployment

 Administration costs: usage and further development or system shutdown The more interesting of these is the administration costs, as it often includes so-called hidden costs. Hidden costs are often only detected in the longer term, but their identifi- cation in advance would be important (West et al., 2004).

3.2.1 Acquisition costs

Acquisition costs include the costs of purchasing equipment and software. In this case, they are direct costs. The hardware includes physical IT devices such as computers and servers. (David et al., 2002) This includes both the purchasing of new equipment and the modification of old equipment for the new use. Purchasing equipment is usually the

(22)

most straightforward part of the process, as the options are leasing or purchasing equipment from a company (Tietotekniikan Liiton Ry 2005). The acquisition cost of the software includes the purchase of the actual software itself and its possible tailoring to fit the needs of the company (Tietotekniikan Liiton Ry 2005). In this connection, the system shutdown discussed in chapter 3.1.5 can also be included, covering the shutdown of the previous system and the transfer of data to the new system from it.

3.2.2 Administration costs

Administration costs are the costs realized in the post-purchase of the system due to a cost from the usage or support of the system. These range from easily measurable capacity costs to more difficult to measure phenomena, such as lowering of work efficiency due to poor usability of the system. (David et al., 2002)

Control costs:

Some system acquisitions show control costs, which are mainly due to the optimization of operational costs. Control includes standardization and centralization of deployment and maintenance. Cost-savings can be achieved by simplifying complex management structures and focusing on fragmented solutions. (David et al., 2002) An example of a cost of a complex system is the increase in training costs. If the system were to be controlled, a simpler system would be easier for employees to work with. (Prior & Zrimsek 2003, p. 2)

Operational costs:

The operational costs are direct and indirect costs. For example, electricity consumption is a direct cost. Indirect costs derive from the system at some point, such as the cost of operating support. This also involves a lot of hidden costs, such as lost working time and its misuse. David et al. divides the operational costs into categories:

 Support Costs

 Evaluation

 Installation and updates

 Training

 Down Time

 Futz factor

 Resource management

 Viruses and power consumption

(23)

(David et al., 2002)

For the purpose of this study, user approvals and system shutdown are further added to the list, which were further discussed in sections 3.1.3 and 3.1.5. In this way, the administrative costs will match to the system life cycle detailed in chapter 2.1. Because of the fact that the operational costs involve a lot of hidden costs, they are going to be further elaborated, so that they can be linked to the total cost of produced by the system.

Support costs come from the end-user's support need for the use of the system. In the initial portions of "Usage”- stage in the system’s lifecycle, the user also needs support for doing basic functions, but at later portions of the lifecycle, the support focuses more on handling issues. In practice this includes, for example, skipping error states with system administrator privileges. (David et al., 2002) Costs also come from maintaining or securing IT support from an external service provider or system vendor (Tietotekniikan Liitto Ry, 2005).

The evaluation costs come from the acquirement of the new system in the form of testing and hours worked. If the system is very scattered, the cost of the evaluation is also higher. Instead, if applications can be used as standard products, the cost is reduced as customization is not required in the implementation or later updates. (David et al., 2002) The cooperation with the supplier of the system should be taken into account and how different agreements can affect the amount of procurement alternatives. Additional con- tractual constraints may lead into additional costs in problem situations. (Tietotekniikan Liitto Ry 2005)

Although the installation and updating costs are largely straightforward, their attachment to other areas should not be ignored. For example, hardware acquisition costs have a very direct impact on installation costs. During the installation of a new system, limitations of existing hardware should be taken into account as they may force the purchase of new equipment. However, by investing in standardized equipment and solutions, installation and upgrade costs can be reduced (David et al., 2002). Centralization in turn sim- plifies the IT infrastructure of the enterprise because then it can be managed from one point. Also by centralizing, the number of different devices and applications decreases.

(David et al., 2002). Cost estimates from external suppliers are often low in order to make offers as competitive as possible. This can later be justified by the fact that it is difficult to estimate the timetables and the hours required for larger systems. (Haddara 2011) This is also due to the fact that the process may manifest unpredictable phases and hidden costs.

(24)

During the deployment phase, particular attention should be paid to the training of employees, that they will be able to benefit from the system and to improve their level of acceptance. Training can take place in two different ways (David et al., 2002). The training session may be a lecture-based or self-conducted training where the user can inde- pendently access the system in controlled mode. If the system is well centralized, training costs are saved because the same training can be given to everyone. Instead, if the system is used on a variety of platforms and through different user interfaces, training must be tailored to each individual group. (David et al., 2002) Saving in training costs is not a good idea because it weakens the benefits gained from the system. According to Haddaran (2011), the total training costs for an ERP system implementation is around 5.9%, which does not include the costs of any possible follow-up trainings needed later on. (Haddara 2011).

Downtime is the time lost when the system use is interrupted, and the system cannot be used. This may be due to technical failures of the system and hardware. For example, a planned upgrade of the system or the installation of a new system that causes a down- turn. Downtime always effects costs for the company regardless of the cause. (David et al., 2002) Costs can be difficult to measure because it may well affect the whole organization's performance. According to a study by CA Technologies (2010), companies lost more than 26.5 billion dollars per year on downtimes. The total cost did not include costs affected by side effects such as the reduction of the company’s image, the morale of employees and the deterioration of confidence. Businesses also felt that a loss of more than 14 hours per year due to down time affected the image and reputation as a result.

Two hundred companies participated in the study. (CA Technologies 2010)

The term Futz factor refers to the indirect costs incurred by the employees through the system. These are not directly related to the system but are due to the way in which employees are using the system or what the employees are using the system for. This is realized as a decrease in efficiency, which in turn generates costs for the organization's turnover. (David et al., 2011) This also includes all the time spent on unnecessarily fin- ishing up work. According to Drury (2001), employees spend about five hours of systems and computers per week for nonwork related activities. (Drury 2001) This also includes independent study of how to use the system and unnecessary monitoring of finished work. (Dryden 1998)

The management of resources includes the examination and management of IT resources as well as the cost factors involved. Costs come from identifying and targeting your organization's physical IT resources. In practice, this often means documentation and a register of the company's physical IT resources. (David et al., 2002) Although

(25)

maintaining documentation will cause a cost, it also facilitates other management tasks because information is readily available.

The last portion of the TCO model presented by David et al. (2002) concerns viruses and electricity consumption. (David et al., 2002) Electricity consumption is straightforward, and this study does not focus on this either. Viruses in turn cause two different costs.

The first is the direct visible costs such as downtime and lost data (David et al., 2002).

The second cost category is not as straightforward but includes the side costs that the virus has caused. These include, for example, the cost of not been able to use the device during the virus, the loss of operational power of the device due to the virus and the time taken to restore the data. (Collier et al., 1993)

As discussed in Chapter 3.1.3, user approval plays a significant role in the deployment phase of the system, so it can also be given value as an indirect influence on the total cost. This can be facilitated by investing in adequate training, thus allowing users to ex- perience higher availability of the system (Morris et al., 2002). Another way of improving acceptance is to involve end-users in the planning process of the system, thus allowing them to contribute to the development of the system. In this way, the end user feels that the system is partly due to his own efforts and input and improving his or her acceptance rate of the system. (Morris et al., 2002) Giving an opportunity for the end-user access to the system enables them to better understand the system and apply its usability to other programs (Venkatesh 2000).

(26)

4. MATURITY MODELS

In previous chapters of the thesis, document management, system life cycle and total costs are studied. This helps understand the current state of the system, and how we have arrived there. This chapter aims to examine ways to evaluate the current state with the use of a maturity model.

Software companies face continuous challenges in maintaining their competitive edge by constantly improving their products and processes. Defining the current state of a process or product is vital to the long-term development and maintenance of it. With the current state defined, it is easier to identify the most vital issues critical to the product’s or process’s quality (Paulk 2002).

Maturity models provide guidance on how to evaluate the current state of development and maintenance processes by assessing them to different levels and elements (Paulk 2002). For the purpose of this study, CMMI was chosen to be the basis for building a maturity model for a DMS. CMMI is a widely used and accepted maturity model, so it was a clear starting point for building a new maturity model.

4.1 Definition of maturity and maturity model

The meaning of maturity can simply be understood as a state where everything is complete, and levels of maturity refer to factors that contribute to this completeness. (Leem et al 2008) SEI (2010) defines the maturity of CMMI-model to come from the constant development of different maturity sub-areas belonging to the model. Fraser et al (2002) states that maturity level concerns either a whole sub-area or a single dimension that has clear differentiating properties and attributes.

Maturity model can be guiding or descriptive, and its use is dependent on which type it is. Generally, it serves as a tool for companies to define their current state, so they better understand their development needs. (Marx, Wortmann & Mayer 2012) The exact method or steps which are used to improve the situation is for the company to find, in other words, the model helps to identify the problem, but does not tell how to fix it. Ac- cording to Leem et al. (2008), the central elements of the model are the model’s states with descriptions and key areas that define how goals of each state are achieved. Cook and Visconti (2008) point out that the maturity model mainly brings out the company’s satisfaction level in areas and not the absolute level. This is very depended on the way

(27)

the level is measured. Levels derived from user interviews are very subjective to user satisfaction, but more technical ways of measuring may give more absolute answers.

Maturity model aims to guide the development process of the organization from one level to another, instead of finding specific problems to solve. For this reason, it is important that the model’s level descriptions are done well enough for the organizations to find themselves in them. (Marx, Wortmann & Mayer 2012) When making or improving an existing maturity model, a Goal Question Metrics (GQM) model can be used. This provides a three-level framework through which the maturity model can be examined: (Cook

& Visconti 1998)

1. Goal: List of principal goals and practices.

2. Question: A question for every goal in part 1, where the answer clarifies if the goal has been reached.

3. Metrics: What must be measured that the question can be answered at a required depth.

The areas that are developed in the maturity model are: the model itself, maturity levels, key practices and questions that rise from these practices. The questions can be simple yes/no questions that can be used to find out if the required level is reached. A scale from 1-5 can also be used to measure how well the goals of an area are reached. (Cook

& Visconti 1998)

4.2 Capability Maturity Model Integrated (CMMI)

The Capability Maturity Model Integrated (CMMI) was originally created from CMM to better support the needs of software development decision making. The model provides a way to show structured information on the organization’s capabilities of software development. The model originates from the capability maturity model that was formed in the late 1990s because of a study that showed that most software development projects failed. (Niazi et al. 2005)

4.2.1 Continues and staged structure

The model does not give an exact maturity model for a DMS, but is a widely used and accepted model, thus giving a good foundation to build upon. Even though the model is for general use, it is important to note the special attributes, processes and business environments. CMMI representation can be shown in a staged structure or in a continues representation. The model’s continues structure is illustrated in figure 7. (SEI 2010)

(28)

CMMI continues structure (adopted from source SEI 2010)

The continues representation is used when the goal is to improve some process area of an organization. Specific and generic goals are defined for each chosen process. The specific and generic practices give the means to reach the set goals. The order in which the improvements are made is not important and they should align with the goals set by the organization. (SEI 2010)

Capability levels are used to define the state of a given process, and their purpose is to describe the evolutionary development path of the process. Each level is made up with generic and more specific practices that determine what level of the process is at. More- over, each level serves as a foundation to the next level, so the positive attributes are cumulative as process moves to higher levels in the model. (SEI 2010) According to SEI (2010), the capability levels that are used in the continues structure are:

 0. Incomplete: The process does not exist or only exists in part; the process does not satisfy all the goals set for it.

 1. Performed: The process fulfills all the set specific and generic practices de- fined for level 1, but the process cannot yield constant positive results. Not all the given objectives are reached such as cost and quality, but some beneficial outputs are received. This is the first true step toward process improvement but does not yet define if the process really works for the organization.

(29)

 2. Managed: As per the levels name, the process is managed by an individual or a group. The process is planned, and the performance is measured. All the goals set by the model are met, and the process actively directs the way things are done in the organization.

 3. Defined: The process is a defined process meaning that a process definition exists for the process. The definition of the process meets the organization’s standard of tailored processes and contributes metrics and other useful data to the organization’s process assets.

 4. Quantitatively Managed: The level 4 process is essentially a level 3 process that is managed using statistics and other quantitative metrics. Statistical goals have been set for the process, and they guide the way the process is managed.

The original goals from early levels, such as quality and costs, have statistical ways to measure them.

 5. Optimizing: A level 5 process includes all the elements listed in the previous capability levels but is constantly improved through the comprehension of devia- tion and common cause found as the process is active. Improvements are made both incrementally and innovatively. The target of improvements is the actual defined process and the standard processes from which it was tailored. Fixes include correcting the actual process definition and training employees, so they can follow the process as intended.

The biggest difference between level 4 and 5 are that level 4 concentrates on improving the process and level 5 is more about improving the whole organization. Similarly, to preserve the highest level of capability, the organization need to improve so the process has room to improve. Even though the model is mostly about improving a single process, the best results are achieved when all the processes are improved at the same time.

(SEI 2010)

The staged structure is similar to the continues structure, but it is more used in the Soft- ware CMM. The structure sets a predefined path for process improvement through defined process areas. Like in the previous structure, each process area so broken down to specific and generic goals that ultimately lead into specific and generic practices. The biggest difference is the use of maturity levels (refer to chapter 4.1) instead of capability levels. (SEI 2010) The staged structure is illustrated in figure 8.

(30)

CMMI staged structure (adopted from source SEI 2010)

The staged representation is ultimately a set of confirmed evolutionary steps toward a more developed process. It also provides a way to compare across organizations through the use maturity levels. All the maturity levels can be sized up to a single rating to further increase the ease of comparison between organizations. To reach a certain maturity level, the organization must fulfill all the set specific and generic goals for that level and all the preceding levels. (SEI 2010) The maturity levels as per SEI (2010) for a staged representation are:

 1. Initial: The process is usually ad hoc, lacks clear order and the environment where it exists is unstable. To be successful, it requires great knowledge and effort from employees. Outputs are received, but with a high cost and low efficiency. Any successes gained cannot be repeated with certainty. This level is like level 0 in the continues structure.

 2. Managed: All the specific and generic goals set for the maturity level are achieved through managing, maintaining, measuring and planning the process.

The existence of the process ensures that practices are performed even during times of stress. Management has insight into the process and can see the status of items in the process. The outputs fulfill set requirements of quality and standards.

(31)

 3. Defined: The process is well understood and documented. The key difference between level 2 and 3 is that in level 3 all the project processes are derived from the same set of organizational standard processes. It remains constant through different projects, while in level 2 the standards and ways of working from which processes are derived from fluctuate between projects. At level 3, the process is tailored from the organization’s standard processes to fit the specific case. Thus, all the processes remain constant throughout the organization, but tailoring is allowed according to the set tailoring guidelines. The process is more proactively managed with the understanding how it aligns with other processes.

 4. Quantitatively Managed: At the fourth level, the key components of the pro- cess are identified and measured statistically. Guidelines and metrics are set for these key components, so their fulfilment can be monitored. It is understood what components give the quality and value to the customer or user. Causes of devi- ation are identified and fixed to prevent their occurrence in the future. The key difference to the previous level is the increase in predictability and transparency to the process.

 5. Optimizing: The final maturity level fulfills all specific and generic goals set from previous levels in addition to the the new specific goals set for level 5. New generic goals are not added after level 3. The process is constantly optimized using the knowledge of common causes of variation present in the process. Like in the continues structure, improvements are made both incrementally and innovatively. Business objectives are constantly reflected upon and changes there are considered when optimizing the process. Any changes done to the process are measured and compared to the state before the change to understand its effect on the key components. While level 4 tries to find individual causes of de- viation, the level 5 concentrates more on finding the common cause behind fluc- tuations and how it effects all the processes involved.

Although the metrics are more abundantly described in the last two levels, it is still important to measure the process in all its stages. This way the effects of moving from one level to another can be measured and justified. A process should consider all the maturity levels because they build upon one another cumulatively. (SEI 2010) Without the disci- pline from earlier levels, the higher levels are prone to fail.

4.2.2 Process areas

As shown in figure 8, process area needs to satisfy a set of goals. The process area consists of correlating practices and need to be implemented and improved together in

(32)

order to fulfill a set goals. A company has to choose from the set of process areas those, which it wants to concentrate on improving. (SEI 2010) Once the areas have been chosen, the desired capability levels for each process area are nominated, so that plans to reach them can be made.

The process areas can be divided into four categories: (SEI 2010)

 Process management

 Project management

 Engineering

 Support

Like shown in figure 8, each process area has specific goals and generic goals. The generic goals main levels as set by the CMMI model are: (SEI 2010)

1. GG1: Achieve specific goals

2. GG2: Institutionalize a managed process 3. GG3: Institutionalize a defined process

4. GG4: Institutionalize a quantitatively managed process 5. GG5: Institutionalize an optimizing process

CMMI model consists of twenty-two process areas. The bolded process areas are considered to be part of the CMMI model for development (SEI 2010)

1. Agreement Management (AM)

2. Capacity and Availability Management (CAM) 3. Causal Analysis and Resolution (CAR) 4. Configuration Management (CM)

5. Decision Analysis and Resolution (DAR) 6. Integrated Project Management (IPM) 7. Measurement and Analysis (MA) 8. Organizational Process Definition (OPD) 9. Organizational Process Focus (OPF)

10. Organizational Performance Management (OPM) 11. Organizational Process Performance (OPP)

(33)

12. Organizational Training (OT) 13. Product Integration (PI)

14. Project Monitoring and Control (PMC) 15. Project Planning (PP)

16. Process and Product Quality Assurance (PPQA) 17. Quantitative Project Management (QPM)

18. Requirements Development (RD) 19. Requirements Management (REQM) 20. Risk Management (RSKM)

21. Supplier Agreement Management (SAM) 22. Technical Solution (TS)

23. Validation (VAL) 24. Verification (VER)

(34)

5. RESEARCH METHODS AND RESULTS

In this chapter, the empirical portion of the study is introduced and the way it was conducted and analyzed. Also, the organization, in which the interviews were done, is introduced. Finally, the results from the interview are presented.

5.1 The target organization introduction

The target organization M-Files is a Finnish technology company that develops and sells information management solutions to the B2B-market. Its service portfolio mainly consists of information management solutions and consulting. The study was done to the department in charge of sales and delivery.

The organizational structure is a project-based matrix organization where employees switch projects and customers regularly. Sometimes bigger customers have many projects and employees might switch between projects within the same customer. As most of the project do not occupy all the time of consultants, often consultants have many projects on-going at the same time. The same employee’s role in different projects vary between being a consultant, sales representative, technical lead or project manager.

5.2 The target software introduction

The product M-Files as a document management solves problems through the use of metadata. Instead of remembering where a document is stored, the users describe the document when saving or finding it. The system utilizes different ways to enable fast finding of documents. Quick searches and more advanced searches can be used to narrow down to the document by describing its metadata to the system. Advanced searches can be stored as views to enable users to sort documents by customers, projects, or any other metadata set to them.

As the documents are no longer in folders but are located in document vault with different ways of finding the same document, only a single copy of each document exists. Check- out and check-in operations make sure that the document is only modified by one person at a time, thus keeping version control in check. The access control can be configured to be automatically calculated from the document’s metadata, thus easing the burden of adding new documents. Also, this eliminates human error as the user is not the one deciding what access control the document should have.

(35)

The document management system integrates seamlessly with Windows file explorer giving the end-user a familiar user interface and controls. Office integration provides the user possibility to move emails easily from outlook and save from office software directly into the system. Sharing documents can be done safely by providing links to colleagues or create temporary URLs to share with customers who do not have access to the system.

Document life cycles are modeled using workflows that give automation and transparency into monitoring documents during their life cycle. Users are notified automatically when their input is needed to advance a workflow, and they can use views to monitor how their documents advance in their life cycle. As each document has a state and a timestamp when they have entered the state, it is possible to find bottlenecks in the process and make reports that show how many documents were in each state at a given time. Further automation can be reached by adding custom code to events that are trig- gered when documents are created and modified.

Even though the system is a product and works as is, it still requires much configuration in order to achieve all of that which was described. In turn, to be able to do the configuration work well, a throughout specification for the system must be made. The end result is solution tailored to a specific customer’s need.

Once the specification and configuration processes are complete, the newest version of M-Files is installed to the customers server, or the customer can optionally use a server based in Azure. As the base M-Files system is the same for all customers, the update process remains the same for everyone as there are no customer specific branches to content with.

5.3 Data collection

Data for the study was gathered by conducting interviews with M-Files employees who have expert knowledge on the M-Files system, delivery model and/or maintenance of existing M-Files customer solutions. As M-Files is rather new entry into the information management ecosystem, using a qualitative method allowed the data collection from subject matter experts that are almost only found within the company of M-Files.

5.3.1 M-Files specialist interviews

A semi-structured interview was held for twelve M-Files specialists. The interviewees were chosen from different backgrounds, so that a wider perspective was given. The different backgrounds were as follows:

(36)

Index Position Role in delivery Background

1 Systems Specialist Configuring and maintaining M-Files Solu- tions for customers

Four years of M-Files ex- perience with over ten years in document management.

2 Systems Specialist Configuring and maintaining M-Files Solu- tions for customers

Five years of M-Files ex- perience with over ten years in software development

3 Technical Advisor Configuring the most complex M-Files solutions and providing support for other consultants.

Nine years of M-Files ex- perience in addition to five years in the IT sector.

4 Solution Architect In charge of defining customer requirements and translating them to M-Files system requirements.

Six years of M-Files ex- perience and five years in document management.

5 Solution Architect In charge of defining customer requirements and translating them to M-Files system requirements.

Five years of M-Files ex- perience and has been the lead architect for two of the largest M-Files enterprise customers.

6 Unit manager Managing consulting teams in charge of de- livering and maintaining M-Files solutions.

Previously work as a Sys- tems Specialist for five years and now has a total of eight years at the company.

Creating a maturity model for a document management system

Klaus Simolin