A Journey of CMMI Level-3 Model Implementation in Product-Centric Agile organization

(1)

A Journey of CMMI Level-3 Model Implementation in Product-Centric Agile organization

Udyant Kumar

Master’s Thesis

Degree Programme in

Information Systems Management 2020

(2)

Abstact

Date

Author

Udyant Kumar Degree programme

Information Systems Management, Master’s Degree Thesis title

A Journey of CMMI level-3 Model Implementation in Product- Centric Agile Organization

Number of pages and appendix pages 82 + 38

Supervisors:

Jarmo Peltoniemi, Haaga-Helia University of Applied Sciences, Helsinki Mikko Rajahalme, Program Manager

Antti Hovi, Program Manager

The objective of this study is to validate the success of CMMI level 3 implementation in an agile product-centric organization. The purpose of CMMI is to make process predictable and mature. It also helps in increasing efficiency, performance and communication of an organization. It is noteworthy that the case company of this study is already using agile practices.

The study uses action research technique as its research approach. The data came from the analysis of multiple projects done within the case company and feedback on CMMI implementation by teams. Multiple data collection techniques used for this study e.g. Semi- structured interview, questionnaires, discussion, presentation, workshop and observation of internal documents. The data was collected by the researcher in collaboration with an EX- TERNAL AGENCY which is monitoring and evaluating the implementation of CMMI in the case company.

The outcome of this study is that the case company has achieved the distinction of CMMI level 3. But the case company is yet to achieve commendable level of impact of CMMI on its operations. The feedback from various teams has established the fact that the case company should perform activities prescribed by CMMI model for next 6-24 months and evaluate its impact. But teams participating in the study are convinced that CMMI will positively impact agile way of working of the case company and improve predictability and performance.

Keywords

CMMI level3, Scrum, Kanban, Agile Values, Agile Principles, Action research, Product-centric

(3)

Preface

This thesis is a final work as partial fulfilment for the degree of Master of Business Admin- istration in Information Systems Management form Haaga-Helia University of Applied Sci- ences, Helsinki, Finland titled “A Journey of CMMI level-3 Model Implementation in Product-Centric Agile Organization”. The content of this thesis is very original and does not infringe any copyright. This thesis took 9 months of rigorous work starting with April 2020 and ending with December 2020.

The author would firstly like to thank to God for his blessings till this thesis research is completed. Author would like to thank his supervisors Dr. Jarmo Peltoniemi, Mr. Mikko Rajahalme and Mr. Antti Hovi for their constant supervision, guidance and useful feed- backs from time to time. Without their support and guidance, this work has not been come to conclusion. The author would like to thank Mr. Naganna Vydyula, and Mr. Prabhu Ra- jagopal for his all assistance in this research work. Mr. Vydyula helped the author to identify problems correctly. Mr. Rajagopal helped the author with access to all documents and activities.

Many people are appreciated for their assistance in data collection process. in this reagard, the author would like to thank people Mr. Hitech Kikani, Mr. Chaitanya Khurana, Ms. Preethi Babu, Ms. Vinodhini Josephine, Mr. Hannu Rytilä, Mr. Mika Lindqvist, Mr. Si- mo Tappola, Mr. Lassi Martala, Mr. Joni Korhonen, Mr. Harri Lukin, Mr. Aleksi Kivini, Mr.

Zhou Wenpeng, Mr. Stephen Jeyaraj, Mr. Johan Grönholm, Mr. Anssi Walder, Mr. Antti Toura, Mr. derick Boamah, Mr. Janne Korkeila, Mr. Joona Saarinen, Mr. Juha Korpi, Ms.

Juhana Leiwo, Mr. Topi Tuulensuu, Mr. Vladimir Tulov, Mr. Johan Backlund, Mr. Juhani Dadek, and Mr. Tero Myllymäki for participating in this research and sharing his opinion.

The author would like to thank his friends for their constant support and motivation during this journey. At the end the author would like to thank his wife, Anuradha and son, Uddish for their love and sacrifices and allowing the author to spend his family time on thesis writing.

Lastly, may be this thesis be useful for readers and for future research about similar topic or any other related field.

Helsinki, December 13^th, 2020

(4)

10 CMMI practice areas and practices for categories 41 11 Capability & Practice Areas Under Doing Category 42 12 Capability & Practice Areas for “Managing” category 43 13 Capability & Practice Areas for “Enabling” category 44 14 Capability & Practice Areas for “Improving” category 45

15 CMMI Practice Areas for Backlog Grooming 48

16 CMMI Practice Area for Continuous Integration 49

17 CMMI Practice Areas for Daily Scrum 49

18 CMMI Practice Areas for Definition of Done 50

19 CMMI PAs for Sprint/Release Planning 51

20 CMMI PAs for Product Backlog 52

21 CMMI PAs for Sprint Demo/Review 52

22 CMMI PAs for User Stories/Epics 53

23 CMMI PAs of Sprint/Release Burndown Chart 54

24 CMMI PAs for Pair Programming 55

25 CMMI practice areas for Refactoring 55

26 CMMI-DEV capabilities 57

27 Impact analysis of CMMI Implementation from Employee Perspective 73 28 Employees view of case company getting benefit of CMMI implemen-

tation

76

(8)

ABBREVIATIONS

Full Name Abstract Form

Engineering and Developing Products EDP

Technical Solution ^TS

Product Integration PI

Ensuring Quality ENQ

Requirements Development and Management RDM

Peer Reviews PR

Verification and Validation VV

Process Quality Assurance PQA

Planning and Managing Work PMW

Estimating EST

Planning PLAN

Monitor and Control MC

Managing the Workforce MWF

Organizational Training OT

Managing Business Resilience MBR

Risk and Opportunity Management RSK

Supporting Implementation SI

Causal Analysis and Resolution CAR

Configuration Management CM

Decision Analysis and Resolution DAR

Improving Performance IMP

Process Asset development PAD

Process Management PCM

Managing Performance and Measurement MPM

Sustaining Habits and Persistence SHP

Governance GOV

Implementation Infrastructure II

Practice Area PA

Product Owner PO

Scrum Master SM

Information Technology IT

Research and Development R&D

(9)

1

1 Introduction

Software industry is a prime example of knowledge industry that focuses on knowledge creation and delivery. In this industry, the knowledge is created as software products or services. The success of software industry depends on adopting a process that can induce productivity, agility and stability. Such process is needed in operation because of perishable or intangible nature of software products or services. The productivity and agility of software companies are measured in terms of quick delivery of high-quality products or services. The process, that can induce agility, productivity and stability, helps the company to deliver quality products quickly and efficiently and helping people to become productive and agile. Now-a-days many companies are establishing themselves in different location of the world for market and employees. Hence, such companies need a process that can help them to meet the challenges arising due to diversity. Such process can become more helpful to the company with operation and markets distribute globally also.

That’s why companies are adopting multiple processes to streamline their operation to meet their goals. The commonly used processes, methodologies, frameworks or tools used in corporate world are Agile Methodology, CMMI, Six Sigma, COBIT, ITIL, TOGAF etc. These processes, frameworks or tools helpful to small and big companies alike and help them to become productive with sustainable growth. That’s why, these processes and tools are becoming de-facto standards for the corporate world. Generally, a company tries to follow these processes, frameworks or tools as per its business needs. These processes, frameworks or tools can be useful in improving quality, productivity, and governance in the organization. It is a good idea that companies spend some time and do some proper research before implementing these processes or tools. The selection of appropriate process or tool helps the company to meet its objectives. Some of these well-known processes, frameworks or tools are briefly discussed as follows.

Agile Methodology is a type of project management process framework that aims to achieve high productivity of distributed and cross-functional team comprised of architects, developers and testers by faster delivery of high-quality products and services. This framework anticipates changes and accommodate them. This way it allows more flexibility than traditional methods. It is possible to make small objective changes without huge amendments to the cost and time. In this regard “Agile Manifesto” proposes a common framework which values interaction and individuals over processes and tools, working software products or services over heavy documentation, customer collaboration over contract negotiation, and quick response to changes. Most commonly used agile processes are Scrum, Kanban, Extreme Programming (XP), Lean Development, Test-Driven de-

(10)

2

velopment, Behavior-Driven development etc. Generally, organizations use a combination of agile practices. Scrum or Kanban is most famous agile process used by organizations.

Scrum is an iterative-incremental process that iterates over 2-4 weeks long cycle. Devel- opment team meet with customers or product owners and prioritize tasks before each sprint. Then. team select a list tasks for the sprint and try to complete it. During sprint, team gathers every day and discuss the progress of project. At the end of a sprint, the team delivers a potentially shippable product. This process is continuously iterated, and the product is incremented with each cycle. Scrum is ideally suited to team working on projects with rapidly changing or highly emergent requirements. This has made agile processes to become very popular in modern businesses particularly in software segment [Beck, K. et.al., 2001; Cockburn, 2002; Schwaber and Beedle, 2002]

CMMI (Capability Maturity Model Integration) is a process and behavioral model that helps an organization to improve process by increasing productivity, process repeatability and increasing efficiency of people. It is not just a process model but also a behavioral model.

This model helps in decreasing risks associated with software, product and service development. It was developed by Software Engineering Institute (SEI) at Carnegie Mellon Uni- versity as a process improvement tool for projects, teams or organizations. It follows an appraisal process that evaluates three specific areas: process and service development, service establishment and management, and product and service acquisition. It is de- signed to improve performance by helping businesses to develop better products and services. It can help business to improve performance by developing measurable bench- marks and tackle risks associated with performance improvement. It provides a structure to encourage productivity and efficient behavior throughout organization. [White, 2018]

SIX Sigma is a set of methods and tools for organizational process improvement. Com- panies are using it to improve operational efficiencies. It improves operational efficiency by using statistical analysis. It uses data-driven methodology to eliminate defects of any business process. A defect can be defined as anything outside of client specifications. Its objective is to reduce the variability in business process. This process helps employees to identify areas for improvement and work towards it continuously by reducing defects and eliminating variations and wastes. The current standard of six sigma process is to reduce defects to a maximum of 3.4 per million or 99.99966% accuracy. [Miller, 2014]

COBIT (Control Objectives for Information and related Technology) is an important framework for IT governance. It is a framework for developing, implementing, monitoring and improving IT governance and management practices. It provides a common language for IT professionals, business executives and compliance auditors to communicate with

(11)

3

each other about goals, objectives, control and results of IT activities. This framework is based on five principles: 1) meeting stakeholders’ requirements, 2) covering enterprises portfolio end-to-end, 3) apply single integrated framework, 4) perform a holistic approach to govern organization, and 5) separate governance from management. In this respect, seven aspects of governance have been identified to support five principles of this framework. These are: 1) principles, policies, and frameworks, 2) processes, 3) organizational structure, 4) organizational culture and ethics, 5) information, 6) services, infrastructure, and products, and 7) people and skills. If there is no framework for common language is used in IT governance then it becomes very difficult for an organization to inform auditors and employees about when, where, how and why specific IT controls are being used. It becomes imperative for an enterprise to devise such plan in practice within organization to justify the need of such a framework which can answer questions related with when, where, how and why specific IT controls should be used. [Moeller, 2013]

ITIL (Information Technology Infrastructure Library) is a tool for IT governance. It provides several benefits such as increased customer satisfaction with IT services, improved service availability that leads to high profit, reduced rework that leads to financial savings, improve time to market for new products and services, improve decision-making and op- timize risk for IT-related processes. It can be used as continuous process. But, ITIL often needs third-party e.g. TIPA (Tudor IT Process Assessment) tools to support its implementation and usage. Compared to COBIT, it has a narrow focus on service management.

COBIT has a broader risk management focus that is applicable to most business areas.

[Moeller, 2013]

TOGAF (The Open Group Architecture Framework) is a framework that can be used with COBIT. It is a governance, risk and compliance framework that compliments COBIT. It is used in designing IT infrastructure adaptable to requirements in an organization. This also provide a common framework that helps easy design and implementation of IT strategy.

[Desfray & Raymond, 2014]

It is highly likely that an organization may need more than single process or framework to improve efficiency and productivity. So, organizations prefer a combination of several processes, frameworks or tools to enhance their operations. In this study the researcher focuses on analyzing the benefit of using a process for an organization already following a selected set of processes. Hence, the researcher will evaluate the compatibility of new process with existing processes and improvement in the operation of the organization.

The case company aims to implement a new process in addition to existing processes to improve predictability and productivity. It is required from the case company to take some

(12)

4

initiative to get desired result. As a part of this study the researcher will evaluate the implementation of a suitable process or tool that can help the case company to get desired result. Hence, the researcher decided to focus on more than single process or framework to get better idea.

The case company is a big multinational product development company located in different parts of the world. Several teams are working at different locations distributed globally.

The case company is having widespread access to global market. Currently more than 15000 employees are working in the company. The main business of this company is help urban life become better. It is building products for multi-story buildings and provide an easy and smooth life to people living in buildings in urban area. Currently the case company is using agile processes that is helping it to become customer centric. The case company aims to have a model that can help it to become process repeatable and predictable.

The case company is looking to implement CMMI with agile process to become more predictable, agile and stable. The benefit of using CMMI with agile is that CMMI tells what to do and agile process explains how to do. Currently the case company knows how to do the things, but it is not sure about doing it in more predictable and disciplined way. Also, the case company is not always sure about its outcome. The different teams of the case company are not following a standard agile process. Coherence between different teams is also missing. Hence, there is a need of a process which can facilitate proper auditing of processes followed by teams and standardize these processes. CMMI can be help the case company to make processes standardize and predictable.

1.1 Research Objective and Context

Every Organization aims to become mature and highly productive organization. The organization can achieve maturity by following well-defined process or framework and align- ing its objectives with productivity. Generally, organizations use a combination of processes or frameworks to increase agility and productivity with stability. But failing to do so it does not become productive and not considered a mature organization. Many reasons can lead to lack of process maturity of the organization. The lack of process maturity can result in failure in project implementation, ignorance about processes, becoming reactive rather proactive, making unrealistic budget and schedule, and sacrificing quality. General- ly, the company does not follow any standards during measurement of quality. (Christie &

Scott, 2005)

(13)

5

The organization can achieve maturity by improvising its processes. The better process can help the mature organization with better inter-group communication and coordination, completion of work within cost and schedule, consistent practices, updating of processes when required, well-defined roles and responsibilities of personnel, and support from management. The organization can become mature by successfully implementing well- defined process model or frameworks in practice. Normally it is expected that process should be characterized for the organization and is proactive. When organizations move to CMMI level 3 then the organization works like clockwork. Work is planned, schedules are achievable, communication occurs well, risks are mitigated, defects are spot on early, and solutions work. Broken process can be easily fixed, and best practices are shared among people. The primary reason for adopting CMMI level 3 within an organization can be having a one-stop place for a complete set of practices that can be packaged into the workflow the organization wants. It also provides a roadmap so that practices can be adopted incrementally to mature the organization over time. Spot on errors and assess risks early. Make the working team more organized. Also get an appraisal that can lead to recognized public rating that demonstrates capability of the organization. (Christie & Scott, 2005)

The software research and development unit of the case company is the area of research of this study. It is likely that the selected process model or framework can affect several practice areas of the company. It can affect practice areas such as systems engineering, software engineering, integrated product and process development, supplier sourcing, and quality assurance. The most common system engineering practice area can contain activities related with configuration management (CM), measurement and analysis (MA), organizational process focus (OPF), project monitoring and control (PMC), process and product quality assurance (PPQA), risk management (RSK), supplier agreement management (SAM) etc. Integrated product and process development practice area can contain activities such as integrated teaming (IT), organizational environment for integration (OEI) etc. Source selection processes can be Integrated supplier management (ISM) etc.

Currently the case company is an agile product-centric organization. But the case company lacks process related maturity and capability. Works done in the organization often lacks clarity and discipline and the outcome is often delayed and poor in quality. The case company aims to implement a process model that can help it to achieve agility with stability and as a result becomes mature and productive. It can help in standardizing processes within organization and work its various units in integrated and cohesive manner. The organization should become proactive and productive. The existing problem with current practices used in the case company is that various units within organization follow practic-

(14)

6

es in different way and lacks uniformity. Currently the process has been defined for each project but does not set common goal for the organization. All units are more reactive than proactive which makes them slow to respond to changes. Hence there is a need of a process model that can integrate the functioning of all units and help them to perform more coordination and integration. The implementation of the process model will help the tar- geted division of the case company to audit its processes and refine their activities. That’s why the case company has chosen CMMI model level 3 to achieve its objective and looking forward to implementing the process model. It will help the organization to define and standardize processes within case company. This will help the company to align various processes within its operation to work for a common goal.

1.2 Research Problems

The case company aims to implement CMMI level 3 in the organization. It is trying to standardize its existing agile processes. In this journey, it is expected that the company will face many challenges. It becomes important for the researcher to identify those challenges and find a way to solve them. In this subsection the most common challenges faced by the case company are being identified and discussed. The researcher will focus on a study to find whether a new process or model can help the case company to get problems solved. The biggest worry for the case company is to find a process model that can help it to streamline its organizational process for higher productivity, efficiency and bring uniformity among different units. The researcher has identified the following problems as a part of his study:

- “How CMMI Model and Agile Methodology can complement each other?”

- “What are benefits of CMMI implementation in a product-centric organiza- tion?”

- “What are challenges and risks associated with CMMI level-3 model imple- mentation in agile organization? How to overcome these challenges?”

To solve the problem faced by the case company, the researcher will work with various teams of the case company and an EXTERNAL AGENCY working on this project. The researcher with EXTERNAL AGENCY will use appropriate research techniques and processes to find the accurate answer of the problem.

1.3 Scope and Limitations

Generally, CMMI presents a process to achieve maturity level 5. But the case company is aiming to achieve CMMI level 3. First, the biggest challenges for the case company is to

(15)

7

establish a defined and standardize process. The company wants to become proactive and mature organization. It wants the process to be defined for the organization at this stage. Hence, the scope for this study is limited to achieve CMMI level 3. Also, this research focuses on studying impact of CMMI implementation current agile practices and challenges of using agile practices in the company. This study will present a holistic view on mitigating the risks associated with CMMI level 3 implementation. The positive side of this study is impact analysis of CMMI level 3 on productivity and agility of the case company. This study will be carried out for the company working in IT research and development and operations mostly.

This study is limited with achieving CMMI level 3. Hence, CMMI level 4 or level 5 are not covered as a part of this study. During this study CMMI will refer to CMMI level 3. Also, this study is more useful for companies working in IT. So, applying the outcome of this study will not be completely applicable to companies working in pure manufacturing sector or service sector. In such cases, further research is required to fine tune to the need of industries different than IT R&D and operations.

1.4 Structure of Thesis

This Thesis has been divided in many chapters. These chapters are Current State Analy- sis, Theoretical Background based on Literature Survey, Conceptual Framework as a part of Theoretical Framework, Validation of Conceptual Framework as a part of Findings, and Discussion and Conclusion.

The chapter on Current State Analysis is focused on study of the current situation of the practices and processes used in the case company. Few teams have been selected as case projects of the study. In this chapter the capability and maturity of selected teams have been studied. The study has been done by performing semi-structured interviews with teams in collaboration with Appraisers and observing internal documents. At the end, the researcher presents gap analysis between goal and current situation as findings.

The chapter on Theoretical Background focuses on literature survey and conceptual framework for the study. The literature survey has been done for the various processes and practices used in the case company. The information has also been collected for the possible impact of using processes and practices. The relevant information has been searched in various literatures and internal documents of the case company. Based on information collected from literatures, internal documents and observation of organization-

(16)

8

al processes, a conceptual framework has been proposed that can help in analysing the outcome of new process implementation and its impact on the case company.

The next chapter on Findings discusses the outcome of new process implementation and its impact in the case company. The impact has been analysed in the form of benefits and challenges on existing processes and challenges faced by various teams during its implementation. The study has been carried out after discussion with selected teams and other stakeholders. The new process implementation has been carried out in multiple iterations.

The last chapter discusses and concludes the study. It compares findings with research objectives. At the end it presents an overview and propose an action plan for the case company. The proposal can be used by the case company for future actions. During the study various things have been discussed such as CMMI compatibility with agile process, usefulness of CMMI in a product-centric agile organization and challenges during CMMI implementation.

(17)

9

2 Research Methodology

This chapter has been divided into four sections focusing on appropriate research approach, procedure, data collection and analysis methods, and validity and reliability of data. It discusses steps for identifying, selecting, processing and analyzing information about the chosen topic. It also helps in proposing a format for critical evaluation of study’s overall validity and reliability.

2.1 Research Approach

In this research, Action Research Method has been used as research approach. This method seeks transformative change by acting and doing research simultaneously. It helps an individual to find solutions to problems encountered in the context which they operate. It is based on systematic and self-critical enquiry to find accurate solution of the problem. Several researchers have defined action research in different way published in various literatures, but all opinions can lead to better understanding of action research method. Some can be presented here:

“Action research is a process of systematic reflection, enquiry and action carried out by individuals about their own professional practice” (Frost, 2002, p. 25)

“Action research is a term used to describe professionals studying their own practice in order to improve it” (GTCW, 2002, p. 15).

“Action research combines a substantive act with a research procedure; it is action disciplined by enquiry, a personal attempt at understanding while en- gaged in a process of improvement and reform” (Hopkins, 2002, p. 42).

“Action research is a flexible spiral process which allows action (change, im- provement) and research (understanding, knowledge) to be achieved at the same time” (Dick, 2002).

“Action research is ... usually described as cyclic, with action and critical re- flection taking place in turn. The reflection is used to review the previous ac- tion and plan the next one” (Dick, 1997).

(18)

10

These above-mentioned definitions help us to understand action research in better way.

The critical evaluation of these definitions helps us to find better meaning of action research method. It can be inferred as a flexible spiral or cyclic process that helps researcher to enquire. It has emphasis on practical way to solve problem which is generally carried out by individuals, or a team of professionals. It involves terms such as research, systematic, critical reflection and action. Such activities are carried out to understand problems, introduce changes and evaluate the outcome. Research often involves gathering and in- terpreting data focusing on continuous action and learning. Critical reflection involves reviewing activities and planning for future activities. In abstract, it can be characterized action research has four characteristics: practical nature, focus on continuous change, cyclic process and involvement of participants. (Coughlan and Brydon-Miller, 2014)

The general research action framework is a cyclical approach that starts with problem identification, data collection, data analysis and feedback evaluation phase. This can also be termed as plan, act, observe and reflect. In Plan stage, researcher develop a research plan based on identified problem and gather relevant information. This plan is proposed for further action. In the Act stage, researcher act on the plan and implement it in the process. In the next Observe stage, researcher collects data based on action plan and ana- lyze this data. He shares the findings of this data with other stakeholders in the program.

In the last Reflect stage, he reflects on the process. If he finds a gap between the out- come and objective, then the plan is revised and proposed for further action. This cycle continues till the objective is met. (Chevalier and Buckles 2013)

Chevalier and Buckles (2013) has presented the basic framework of action research method as follows:

Figure 1. A basic Action Research Model (Chevalier & Buckles, 2013)

This model can be further extended to incorporate more methodical, and iterative approach embracing problem identification, action planning, action implementation, outcome

(19)

11

evaluation, and incorporate reflection. In this extended form of action research model, people can use various cycles to plan and incorporate changes and further evaluation to reach an outcome. The model proposed by Carr and Kemmis (1986) can help us to understand the concept of action research cycle as continuous learning process where people can learn and create knowledge by critically reflecting upon their actions and experi- ences. During the process participants can form abstract concepts and test these concepts in next iterations. This way participants can create their own understanding of situation and act accordingly. This will help them to improve practice and advance their knowledge in chosen field. The extended form of action research model proposed by Carr and Kemmis (1986) is presented as follows:

Figure 2. An extended Action Research Model (Carr & Kemmis, 1986)

The researcher has used the model proposed by Carr and Kimmis as the basis of research technique. The research has been carried out in several cycles and reviewed at the end of cycles. Later, findings after reflections have been accommodated in next cycle.

2.2 Research Process

The research process of this study is based on action research and includes a series of steps required to solve the research problem. The flow includes both stage-wise and iterative process in the study. The flow diagram below presents an overview of the research process used in the study.

The process uses several steps defined as: Identification of Objective, Current State Analysis, Literature Survey, Conceptual Framework Design, Initial Model Evaluation (iterative process), and Final Proposal. All steps can be described in detail here.

(20)

12

Figure 3. Research Procedure of this study.

(21)

13

The first step of action research process is identification problem. In this step, the researcher identifies the problem of the case company. At this point, the researcher defines the objective of this research and enlists problems for the research.

In the second step, the researcher does current state analysis of the case company. The researcher evaluates the current practices and processes used by the case company.

Based on data the researcher evaluates strengths and weaknesses of the case company.

The current state analysis justifies the need of research and supports research problem of the study.

The third step is Literature survey. Based on research objective, problems and current state analysis of the case company, the researcher gathers information from various literatures and internal documents used of the case company. At this stage, the researcher gathers information related with processes used in the case company and steps required to solve research problem.

The fourth step of the research procedure is presenting a conceptual framework of this study. The conceptual framework is based on findings from both current state analysis and literature survey. This step forms the theoretical backbone of the research. This framework is discussed with selected team’s representatives and other stakeholders and used for study.

The fifth step is discussing the conceptual framework with selected team’s representatives and analyzing outcome. The conceptual framework is followed by selected teams with a cycle of planning, acting on the plan, evaluating the outcome and providing feedback. This process can run in several iterations depending upon the goal. The cycle ends when the team meets the objective. In this step the data is collected and analyzed at the final stage.

During the implementation of the process, teams are suggested to perform stages of plan, do, check and reflect. This will help teams to correct its mistakes. The researcher and other appraisal team members can help teams to correct their mistakes and adjust accordingly.

The last step of the research is impact analysis after process implementation. Based on findings the researcher can propose some guidelines. these guidelines can be used by the case company to take further measures. Also, this study can be used as a case study for other companies while implementing the similar process.

(22)

14 2.3 Data Collection and Analysis

In this study the qualitative research technique is used. Qualitative data collection methods e.g. semi-structured interviews, observation of documents, questionnaires, and discussion with focus group can be used in this research. Some other techniques such as measurement of impact and consequences can be used for study also. The measurement of impact and consequences will be done quantitatively.

In this study, the data has been collected from various sources. The data has been collected from team’s representatives and members by semi-structured interviews, discussion and questionnaires. Also, observation of internal documents has been a part of this study. The researcher has selected three teams as part of this study. The data has been collected from teams at different times in this study. The teams are located both in Finland and India. Hence, the selection of teams adequately represents the functional and geo- graphical distribution across the globe.

The data collection of selected teams at first stage is presented in Table 1. At this stage data is collected for current state analysis. The first stage of data analysis was done by appraisal team and the researcher.

Interview Participants Date Duration Documented Case

Project 1

MR (PM) & HK (PO) and his team (4 members)

26.4.2020 1 hr Appendix 1 &

Appendix 4

Case Project 2

MR (PM) & HR (PO) and his team (9 members)

26.4.2020 1 hr Appendix 2 &

Appendix 4

Case Project 3

JR (PM) & JG (PO) and his team (14 members)

27.4.2020 1 hr Appendix 6 &

Appendix 4

Table 1. Data Collection Stage 1 for Current State Analysis

The second stage of data collection is presented in Table 2. At this stage the data is collected and analyzed after Final appraisal. The second stage of data analysis was done by appraisal team including the researcher.

Interview Participants Date Duration Documented

1 HK (PO) & MR (PM) 2.11.2020 1 hr Appendix 7

2 HR (PO) & MR (PM) 2.11.2020 1 hr Appendix 7

(23)

15

3 JG (PO) & JH (PM) 3.11.2020 1 hr Appendix 7

Table 2. Data Collection Stage 2 for Final Appraisal

The third stage of data collection is used for impact analysis post CMMI implementation in the case company. At this stage the data has been collected from the same team. The third stage of data analysis was done by the researcher individually.

Interview Participants Date Duration Documented 1 HK (PO) and his team

(4 members)

18.11.2020 1 hr (discussion)

Appendix xx

2 HR (PO) and his team (9 members)

25.11.2020 1 hr (discussion)

Appendix yy

3 JG (PO) and his team (14 members)

19.11.2020 1 hr (discussion Appendix zz

Table 3. Data Collection Stage 3 for Impact Analysis post CMMI Implementation

The following table presents an information about appraisal team which performed appraisal of CMMI implementation and workshop team that was involved in reviewing and approving conceptual framework proposed by researcher. The appraisal team was formed of people of EXTERNAL AGENCY, case company and the researcher. The appraisal team worked with selected teams of case company to implement CMMI. The appraisal team reviewed and gave feedback for improvement in CMMI implementation. The appraisal team used benchmarking appraisal for CMMI implementation. The workshop team reviewed and gave feedback on conceptual framework developed by the researcher. The workshop team was formed of people from EXTERNAL AGENCY and case company.

People Purpose Techniques used Duration

From EXTERNAL AGENCY:

SV (Lead Appraiser), SP From Case Company:

BK, MD, MR, MAR, SVS, SS, TH, AV, PR, UK

Appraisal Benchmarking,

Observation of Documents, Semi- structured interview, Presentation

10.08.2020 – 27.10.2020

PR, MR, AV, SP, HK, HR, JG Workshop for conceptual framework analysis

Presentation, Dis- cussion, Review and Feedback

10.08.2020

Table 4. Teams for Appraisal of CMMI implementation and Workshop for Conceptual Framework Analysis

(24)

16

The data collection, presented in Table 5, has been done in the form of feedback. The feedback has been presented in the form of recording and documentation of observations for different case projects.

Case Projects Duration Contents Observation Log Case Project 1 26.4.2020

2.11.2020 18.11.2020

Semi-Structured Interviews, Observation of internal documents and artefacts used in the team, Discussion, Presentation and Questionnaires

audio, field notes, internal documents repository

Case Project 2 26.4.2020 2.11.2020 25.11.2020

Case Project 3 27.4.2020 3.11.2020 19.11.2020

Table 5. Observation of Data Collected from teams of the case company

In Table 5, the various processes used by different case projects is documented. This list presents an overview of Process Models with specific purpose used by different case projects.

Case Projects Processes and Tools Used

Purpose Contents

Case Project 1 Scrum, Kanban, Version1, Jama, Confluence

Documentation, Sprint activities with Kanban board, Defect log- ging, Content management

documents, notes

Case Project 2 Scrum, Version1, Jama, Documentation, documents, notes

(25)

17

Confluence, SharePoint Sprint activities with Kanban board, Defect log- ging, Content management Case Project 3 Scrum, Version1, Jama,

Confluence, SharePoint

Documentation, Sprint activities with Kanban board, Defect log- ging, Content management

documents, notes

Table 6. Processes and Tools used by the case company in Different projects

The number of sources for data collection are enough for researcher to analyze the process implementation and its impact in the case company. The different techniques used in data collection also helped the researcher to have thorough investigation of the process implementation and its consequences. Since the number of case projects is limited to 3 but people participating in the data collection is significantly enough in number. Hence the data collection can be said appropriate.

The researcher is working in the company, but he is working with EXTERNAL AGENCY in data collection and analysis. Hence, the research outcome is not influenced by researcher being a part of the company. The researcher conducted various activities such as collect- ing data from key the personnel of the case company and relevant information from literature.

In this study the impact of implementation of CMMI in the case company has been done with the same group of people from Table 1 in the form of questionnaires to make sure that the outcome can be truly represented.

2.4 Validity and Reliability

Validity and Reliability of data is an integral part of qualitative research techniques such as action research. It is required that research should be generalizable. It means that the outcome should be replicated anywhere and informative in all situations. Although in action research generalizability of data is not an important criterion hence the common tools to measure validity and reliability of data is not applicable. Mills (2000) has properly defined validity and reliability of data in action research.

(26)

18

Generally, Validity is associated with the measures which is used in data collection. It is concerned with the measure which is used in accurate measurement of data. The common processes are followed while measuring validity of data are information gathering and data review by experienced personal. There are five criteria for validity of action research: Democratic validity, Outcome Validity, Process validity, Catalytic validity and Dia- logic validity. Democratic validity asks, “whether the various perspectives of participants have been accurately represented in the study”. Outcome validity asks, “whether the action as outcome emerging from study leads to successful resolution of the problem”. Pro- cess validity asks, “Is the study being conducted in a dependable and competent manner?”. Catalytic validity asks, “whether the outcome of the study works as a catalyst for future action”. Dialogic validity asks, “whether the research process and outcome have been reviewed by peers and experts”. Hence, validity points out the importance of the tool being used in measurement of study and the understanding the worthiness of the study.

(Mills, 2000)

Reliability always comes along with validity. It ensures that the data has been collected over a period and by researchers are consistent in nature. It answers the questions that the test or survey will produce the same result if done at different times with different set of teams. It helps in testing the measurement technique used in data collection and analysis. Generally, it is a challenge for any group of researchers because the data collection method cannot be retake. In case of action research, it is required that data collection techniques should give good, consistent and reasonable information from those who are using it. (Mills, 2000)

But ensuring validity and reliability of data collection and analysis is often a major challenge and associated with some risks. Brink (1993) has pointed out some risks or errors associated with data collection techniques used in the study. There is high likelihood that researcher can be biased during research and can make some presumptions that can lead to erroneous data. Also, the participants can have presumptions which can result erroneous data. Also, the situation such as geography, culture, work habits, etc can affect the consistency of data. Finally, the poor and inappropriate data collection techniques can result in inconsistent data. So, the researcher participating in the study should focus on the risks associated with the study and plan to mitigate it.

(27)

19

3 Current State Analysis

This chapter focuses on current state analysis of the case company. The benefit of current state analysis is to nail research scope, focus and capture issues and priorities, visualize the work and uncover challenges of the business, and develop a baseline to measure improvements. According to McKay (2019), current state analysis is key of any research to understand the key steps to be taken in building the case company’s awareness for the need of change. It helps in identifying the gaps between the current state and future states. It is very difficult to embark on a journey of change without a baseline. In such case the solution does not address the key business challenges. The current state analysis helps in defining the problem correctly. As Steve Jobs rightly said:

“If you define the problem correctly, you almost have the solution.”

This chapter is divided into several sections focusing on current state of selected case projects. The emphasis is on understanding current processes, methods and tools used currently in case company. The selection of case projects is based on functional and geo- graphical distribution of various teams in the organization.

3.1 Overview of Case Company

The case company is an international engineering and service company headquartered in Finland. It was established in 1910 and currently employ more than 55000 people globally. The case company is engaged in building and servicing products related with urban life improvement. The product or services are used in building construction. It has operations in more than 50 countries with more than 1000 offices. It provides local services to house builders, owners, designers and architects. The company decided to move implement digital experience to its products and services.

The study has been conducted for software research and development division of the case company. Three teams have been selected for this study. All teams are operating both in Finland and India. The selection of teams is based on distributed location, work culture and team composition. These teams are CT Team (case project 1 from Finland and India), AP Team (case project 2 from Finland and India), and KCE Group Team (case project 3 from Finland). This research has been carried out in collaboration with EXTER- NAL AGENCY which is helping the case company in implementing CMMI model. The data collection and analysis for the current state analysis has been done in collaboration with

(28)

20

the EXTERNAL AGENCY. Data is being used in this study with the consent of the EX- TERNAL AGENCY and the case company.

The CMMI model implementation in the case company was started in January 2019. So far three rounds of implementation review have been completed. The implemented review gives an idea about teams achieving the percentage of objectives of CMMI implementation. The success is measured in terms of progress made by teams in different practice areas. The current state analysis of this Thesis will be based on findings after third implementation review. In this chapter, the researcher will discuss the findings implementation review 3 (IR3). Hence, IR3 will be used as the basis of current state analysis of the case company in this study. In next subsections, the findings of implementation reviews will be discussed for selected teams describes as case projects.

3.2 Case Project 1

The team selected for Case project 1 is CT Team which is based in Finland and India.

Here, the researcher discusses the current state analysis of Case Project 1. This discussion focuses on practices, frameworks or tools used by the team. For this study, the data has been collected and analyzed using qualitative techniques e.g. interviews, group discussion and observation of documents and activities. The findings are documented in Ap- pendix 1 and 4. The current state analysis has been performed based on Implementation Review 3 (IR3).

Current state analysis of this case project reveals that the team is currently using agile practices (both Scrum and Kanban). The team follows Scrum activities such as Sprint Planning, Backlog Grooming, Daily Scrum, Sprint Review, Sprint Demo and Sprint Retro- spectives. It follows Cumulative workflow for monitoring its performance. The team is trying to implement CMMI level 3 with agile practices to improve performance and bring predictability. After third implementation review of CMMI implementation, it has been found that the team has not achieved success and need to improve in many areas of agile practices. Hence the team is not able to achieve CMMI level 3. Still the team need to bring repeatability, maturity and stability in its activities. Hence, it cannot be said that the team has achieved CMMI level 3 capability and maturity. The team cannot ensure the business stakeholders that the team can achieve its business objectives repeatedly. The team lacks some capability to meet the criteria of both organization-centric global practice areas and team-specific practice areas. The current state analysis of the team can be explained with strengths and weaknesses of practices followed by the team. In the following paragraphs the strengths and weaknesses of the team are being discussed.

(29)

21

After third implementation review, the team shows capability and maturity in some practice areas. The team is strong in practice areas such as PLAN, RDM and TS. There are some reasons that helps the company to show capability and maturity in the above-mentioned PAs. The team uses a tool (Confluence) to store vital information such as integration testing process, Dev, Integration and Release environment details, and master links to all processes. The coding standards and guidelines to write test scripts has also been documented and followed in practice. This information is shared with other teams. The focus of documentation is that all necessary information should be readily made available to all stake holders for reference and usage. It has been ensured that the requirement can easily be traced back. The team has been advised to continue the above-mentioned practices repeatedly.

But the team does not show significant capability and maturity in some other practice areas. The team needs to improve in the area of Sprint planning (PLAN), monitoring and Control (MC), managing performance and its measurement (MPM), and peer review (PR) of development particularly code. The team does not break user stories into tasks with corresponding efforts efficiently. Time estimation for task is also poor. Hence, remaining work to be completed at any point of time is not known. Also, significant delays are not identified early for appropriate actions. Hence, the team is not efficient in sprint planning.

Further, the team does not justify that its solution meets customer expectations. It does not produce any evidence of any mechanism that verifies coverage of test scripts to product functionality. The team has not demonstrated the current status, risks and impedi- ments in progress review meetings with management. The best tool in such case is KPI dashboard. But there is no evidence of inference drawn from KPI dashboard to identify areas of improvement. Henceforth, it may not be possible to identify significant delay in deliveries and take timely corrective actions. This way the team may not be ensure that performance objectives has been met in its case. The team also conducts Sprint Retro- spectives regularly. But there is no evidence of focused action items towards process improvements. Also, the team does not initiate corrective actions for process improvements on time. Such feedback points out that the team lacks proper actions in certain process areas. The team has been advised to improve in the above-mentioned PAs in the following sprints.

3.3 Case Project 2

The Case Project 2 of this study is AP Team located in Finland and India. In this subsection the current state analysis of the team is being discussed. The data has been collected

(30)

22

and analyzed using qualitative techniques such as interviews, group discussion and observation of documents and activities. The findings are documented in Appendix 2 and 4.

The current state analysis is based on Implementation Review 3 (IR3).

Like previous team, this team also uses agile process (Scrum). The team is capable and mature in few practice areas but needs to improve in some other practice areas. The strengths and weaknesses of the team are being discussed in the following paragraphs.

The team has made some improvements in RDM and PLAN. The team uses a tool (Con- fluence) to store vital information such as integration testing process, Dev, Integration and Release environment details, master links to all processes. Also, coding standards and guidelines are used in writing source code and stored in confluence page. Clarification or queries related to requirements are stored in confluence with details of the query, response, points discussed, and any subsequent decision taken. It ensures that all necessary information is captured to remove ambiguity and provide deeper understanding of requirements. The team is able to do sprint planning and task creation with time estimation correctly. Hence in such cases the team does not require much improvement.

But the team still need to show improvements in some other practice areas such as RDM, EST, TS, PR, MPM and RSK. Effort estimation is not done correctly for few tasks. That’s why it may not be possible to identify significant delay in deliveries and take timely corrective actions. Requirements specification and Technical specification review comments are not captured and tracked to closure properly which do not ensure that all review comments are tracked to closure. Requirements should be written in some tool and linked to test cases. There is no evidence of inference drawn from KPI dashboard to identify areas of improvement. The improvement actions discussed during daily standup have not been tracked properly. So, it may not be ensured that performance objectives are met, and all actions are tracked to closure. Retrospectives are conducted on a regular frequency, however mechanism to capture focused action items towards process improvements needs to be strengthened. There is no evidence of structured risk handling, although few risks are identified during Quarter planning and weekly progress meetings. This may not ensure appropriate handling of risks which may in turn impact the releases. There is not efficient mechanism or use of appropriate tool to do peer review of code or deliverables.

There should be use of tool for this activity to enhance PR practice area. This will help better code quality and team efficiency.

(31)

23 3.4 Case Project 3

The Case Project 3 discusses current state analysis of KCE Group Team located in Fin- land. The data has been collected and analyzed using qualitative techniques such as interviews, group discussion and observation of documents and activities. The findings are documented in Appendix 3 and 4. The current state analysis has been performed based on Implementation Review 3 (IR3).

This team follows agile practice (Scrum). The current state analysis of this team shows capability and maturity in certain specific PAs but expecting to improve in some other PAs.

Hence the team has been advised to improve its weaknesses to achieve CMMI level 3.

The strengths and weaknesses of the team are being discussed as follows.

The team shows improvement in the area of PLAN. Sprint planning is done properly using presentations, prioritizing tasks, breaking stories into smaller parts, detailing features and defects, and complying with the requirements of the release. This provides more clarity to scrum team on the goals to achieve and get the team committed to its task. Developer demo meeting are conducted for knowledge sharing sessions with the team. It helps to upskill the knowledge level of the team members.

But the team needs to make much improvement in the area of EST, RDM, MPM, VV, DAR, CAR and CM. Story point estimation does not follow any standard technique e.g.

Fibonacci series. Hence, the size of the story cannot be accurately estimated. No evidence between user story to corresponding requirements in specific tool. This cannot ensure consistency between the requirements and the solution. This can result in customer dissatisfaction. Requirement review comments are not formally captured. This also does not ensure that all review comments are properly closed. No traceability can be found related with capture and response of clarifications related with requirements. Inference drawn from KPI dashboard is missing. Hence, it may not be ensured that performance objectives have been met and the team has made improvement. Root cause analysis is also lacking any evidence of corrective actions. This does not ensure that the process effectiveness for the team has improved over a period of time. Action derived from root cause analysis are also not established as process improvements. Effort estimation is also not done correctly in some cases. Criteria for evaluation of alternative approaches and the selected outcomes cannot be demonstrated and henceforth probability of selecting an optimal solution cannot be ensured for critical decisions. The tool used to store various artefacts is also not used properly. So, it may not be ensured that right version of

(32)

24

information is available on demand. Hence, the team needs to improve in these areas by practicing repeatedly.

3.5 Summary

The analysis of three case projects presents an overall picture of the organization. It has been found that these three teams as a part of the organization need to follow agile practices in more consistent and cohesive manner. They need to perform activities repeatedly so that that can bring more consistency. If the organization aims to achieve CMMI level 3 then all its units need to conform to the agile values and principles. It is clear that teams are following agile practices (Scrum and Kanban). They follow scrum activities such as Sprint Planning, Backlog Grooming, Daily Scrum, Review, Demo and Retrospectives.

Teams are using Velocity or Cumulative workflow for monitoring its progress. But the current state analysis reveals another story. Most teams are not following agile practices that can be easily traced and monitored. This does not ensure that the organization can show maturity and capability to demonstrate its activity. It is very difficult to trace the compliance followed by teams. It becomes very difficult to audit the activities of the organization. It cannot be ensured that all teams of the organization are working in unison. So, it cannot be guaranteed that the organization is following any standard process for its activities.

Appendix 5 and 6 summarizes the current state analysis of the organization. In Appendix 5, the weaknesses of teams in specific PAs and its impact has been documented. Appen- dix 6 presents a gap analysis between current state and objective of CMMI implementation. Appendix 5 and 6 can provide guidelines for building conceptual framework and subsequent actions to achieve CMMI level 3.

The objective of the organization is to implement a model that can help teams to standardize processes. The model should also help in integrating activities of various projects and teams. This study has used three implementation reviews but requires some more reviews and benchmarking to achieve the target. If the organization is labelled with proper model, then it can help teams to synchronize its activities as per standards. The progress up to third implementation review has underlined the fact that teams have made much progress but requires some more actions to synchronize further and achieve the objective.

It can be concluded that the decision to implement CMMI level 3 is a much-needed model in practice, but it needs efforts from teams to reach the level.

A Journey of CMMI Level-3 Model Implementation in Product-Centric Agile organization