Anisul Islam
EVALUATION OF OPEN SOURCE PROJECT MANAGEMENT SOFTWARE
Faculty of Information Technology and Communication Sciences M. Sc. Thesis Examiner: Dr Zheying Zhang Examiner: Sergio Moreschini (Doctoral Researcher) August 2021
Anisul Islam: Evaluation of Open Source Project Management Software M.Sc. Thesis
Tampere University
Master's Degree Programme in Software, Web & Cloud August 2021
In many areas, Open Source Software (OSS) demand is growing rapidly in the current software business since its quality rivals closed source software. An effective open-source Project Management Software (PMS) plays a significant role in making a project successful. It is always difficult to choose and evaluate open-source PMS from many alternatives. Lack of technical understanding, many options for solving a specific problem and the lack of universally agreed criteria for assessment are some of the difficulties. The research aims to provide a method that does not require deep knowledge and intense performance testing to assess an effective open- source PMS.
From the literature review, the research found twenty-one criteria for selecting an OSS. Such criteria for OSS selection were used to build the foundations of the developed method for assessing open-source PMS more effectively. Then we surveyed a total of 640 students and developers to find out the most influencing factors when assessing an open-source PMS. The Likert five-point scale helped to assess the influence of each variable of the survey.
Survey results indicated that "Security" is the most influencing criterion for selecting an open- source PMS. "Performance" criterion ranked second where "Usability" and "Functionality" are ranked third as their mean score is identical. The research also found that "Human hierarchies"
is the least important criterion for selecting PMS. Finally, the "Licensing" criterion ranked as the third most essential criterion according to developers opinion. However, participants of other occupations did not score high for the "Licensing" criterion due to the lack of understanding of the OSS licensing policy, and therefore the overall ranking was 14th.
The research develops a system for assessing open-source PMS that does not require high technical skill and intensive performance testing. The findings demonstrate the most important factors that end-users, especially students and developers, consider when assessing open- source PMS. In addition, the criteria we discovered in the literature analysis is useful to evaluate any open-source software, not just PMS.
Keywords and terms: open-source software, project management software, assessment methods, evaluation process, evaluation criteria, assessment criteria, assessment, selection
The originality of this thesis has been checked using the Turnitin OriginalityCheck service.
First and foremost, I am grateful to Allah, my creator, for raining His blessings on me during my research endeavour, allowing me to complete the project successfully.
Dr Zheying Zhang, my thesis supervisor, deserves my sincere gratitude for her continued support, motivation and encouragement. Her advice was invaluable throughout my re- search and thesis writing. For my master's thesis, I couldn't have asked for a more quali- fied advisor and mentor.
Aside from my supervisor, I'd want to express my gratitude to my university's admin- istration for their assistance. My profound gratitude goes to my course coordinator, Kirsi Tuominen, for her endless guidance from the beginning to the end of my time at Tampere University. I truly admire her patience and dedication towards me over the years.
My sincere thanks to all the Tampere University teachers and teachers from Bangladesh:
Alak Kanti Sharma, Md. Sohidur Rahman, Rumel M. S. Rahman Pir, Md. Asaduzzaman Khan, Selina Sharmin, Minhazul Haque Riad. I am also thankful to my classmates:
Jahedur Rahman, Md. Emadur Rahman for their support during my thesis.
Aminul Islam and Sufia Begum, my parents, deserve my thanks for their love and sacri- fices in raising and teaching me. My heartfelt thanks are out to Rabeya Begum, my lov- ing, supportive, and caring wife. Her support through difficult times is much appreciated and acknowledged. Knowing that she is willing to oversee our household activities while I finished my work gave me great comfort and relief.
Finally, I would like to express my gratitude to everyone who has helped me with my research.
1 Introduction ... 1
1.1 Problem Definition 2
1.2 Research Aim 2
1.3 Research Questions 3
1.4 Research Methods 3
2 Open Source Software ... 5
2.1 Types of Software 5
2.2 License Policies of OSS 10
2.3 License Compatibility 12
2.4 Summary 12
3 Open Source Software Assessment Models ... 13
3.1 C-OSMM 13
3.2 N-OSMM 19
3.3 Open BRR 22
3.4 E-OSS 27
3.5 Comparison of Assessment Models 32
3.6 Model Development 35
4 Research Methodology ... 37
4.1 Project Management Software 37
4.2 Research Design 40
4.3 Objectives and Questionnaire Design 41
4.4 Validation of the Questionnaire 43
4.5 Data Collection 45
5 Survey Results and Discussion ... 46
5.1 Data Analysis and Results 46
5.2 Discussion of the Key Results 54
6 Conclusion and Future Work ... 57 7 REFERENCES ... 58 8 APPENDICES... 62
Figure 1: Categories of software ... 5
Figure 2: A categorization Of OSS license types, their main properties, and some characteristic examples. ... 11
Figure 3: License compatibility diagram. ... 12
Figure 4a: Product indicators of C-OSMM model ... 14
Figure 4b: Product indicators of C-OSMM model ... 15
Figure 4c: Product indicators of C-OSMM model ... 16
Figure 5: Application indicators of C-OSMM model ... 17
Figure 6: C-OSMM process ... 19
Figure 7: Default OSMM element weightings. ... 21
Figure 8: The four phases of the Open BRR model ... 22
Figure 9: The criteria of the open BRR model ... 23
Figure 10a: Metrics and scoring of the Open BRR model ... 24
Figure 10b: Metrics and scoring of the Open BRR model ... 25
Figure 11: Four stages of the E-OSS model. ... 27
Figure 12: Different categories of identification phase of E-OSS phase. ... 28
Figure 13: Metrics of the product group. ... 30
Figure 14: Metrics of the quality group. ... 30
Figure 15: Metrics of the integration group. ... 31
Figure 16: Metrics of the facility group. ... 32
Figure 17: Comparison of OSS maturity models. ... 33
Figure 18: Criteria of C-OSMM, N-OSMM, Open BRR, and E-OSS model. ... 34
Figure 19: A flowchart of the assessment process. ... 36
Figure 20: The overall structure of research methodology... 40
Figure 21: Criteria affect assessing an OSS ... 43
Figure 22: Participants understanding of OSS. ... 48
Figure 23: Importance of OSS. ... 49
Figure 24: Interest towards OSS. ... 49
Figure 25: Information sources of OSS. ... 50
Figure 26: Participants understanding of OSS license policy. ... 50
Figure 27: Reasons for using open-source PMS. ... 51
Figure 28: Selection process of open-source PMS. ... 51
Table 1: The difference between OSS and proprietary software... 10
Table 2: Key features of 5 open-source PMS ... 37
Table 3: Likert scale for data analysis ... 46
Table 4: Participants sample by geography ... 47
Table 5: Participants sample by current occupation ... 47
Table 6: Participants sample by highest educational qualification obtained ... 47
Table 7: OSS development experience of the participants ... 48
Table 8: Most influential criteria for assessing open-source PMS ... 53
BSD Berkeley Software Distribution
C-OSMM Capegimini Open Source Maturity Model E-OSS Easiest Open Source Selection Model FOSS Free Open Source Software
FSF Free Software Foundation GPL General Public License
N-OSMM Navica Open Source Maturity Model Open BRR Open Business Readiness Rating OSMM Open Source Maturity Model OSS Open Source Software
PMS Project Management Software
1 Introduction
Open Source Software (OSS) increases with great prominence in the modern software industry as its quality competes with closed source software in many cases (Sbai et al., 2018). An effective open-source Project Management Software (PMS) can give team members and other stakeholders a clear picture of who is doing what and where a particular job is in the process (Wilkes, 1987). However, assessing an open-source PMS out of many alternatives has become a real challenge for the users. SourceForge.net, one of the most well-known free hosting sites for OSS, reported hosting over five hundred thousand projects and more than three million registered users (Kamal et al., 2018).
Anyone can create an OSS and then host it on SourceForge.net. This low entry threshold has increased immature OSS projects and added additional challenges for the assessment process (Deprez & Alexandre, 2008). In addition, limited documentation is a barrier to assessing OSS projects (Lenarduzzi et al., 2020).
OSS is accessible publicly to modify and share. Some well-known applications and platforms, such as Linux operating systems (Linux.Org, n.d.) and VLC media players (VideoLAN, 2021), are the best example of OSS. OSS allows developers to inspect, copy, modify the existing project and create new projects. OSS is different from closed or proprietary software such as Microsoft Office (Office 365 Login | Microsoft Office, 2018).
Only the author has the legal rights for alternating, inspecting, and copying the source code in proprietary software. There are different types of license policies that need to be taken into account when dealing with OSS.
Many advantages, such as easier customisation, trialability and cost savings, persuade stakeholders, such as government agencies, to embrace OSS (Mijinyawa & Abdulwahab, 2014). However, the most significant positive aspect of the OSS project is the availability of source code to review, which is essential to measure a software's quality and maintainability (Ahmad & Laplante, 2011). Even though the low cost is a significant advantage, a survey conducted by computer economics shows that less dependence on vendors is the most influential factor in using OSS (Scavo, 2015). On the other hand, some of the major disadvantages of OSS are lack of proper support, insufficient documentation, platform dependency, low-quality code for immature projects. However,
The drawbacks of OSS are not an obstacle for the stakeholders considering the positive aspects.
1.1 Problem Definition
PMS aids in organizing, tracking, and carrying out work on a project. It can assist in managing costs, budgets, time, collaboration and communication during a project life cycle. Business globalization and worldwide mergers are increasing the demand for collaboration. Collaboration can take up a lot of time for project workers if they do not have the right PMS. The success of a project highly depends on an appropriate open- source PMS. Unfortunately, the end-users, such as students, teachers, developers, or anyone who wants to manage projects, always face difficulty finding an OSS to fulfil their project management needs. Most end-users are unaware of the OSS license terminology, which is critical to assess open-source candidates. Many options for solving a particular problem have increased the decision's complexity. Assessing the right OSS for a specific requirement is always challenging as no universally accepted criteria for assessing software. Evaluators frequently employ ad hoc, readily available criteria for evaluation, which is not repeatable and slows down the development process (Ahmad &
Laplante, 2011).
On top of that, many existing assessment methodologies for assessing OSS, including PMS, are not flexible enough for calculating the OSS project's score. Some of them are time-consuming and not straightforward about their metrics for measuring criteria. Some existing assessment models consider functional criteria, while others place a greater emphasis on non-functional criteria. A few models are mathematical, while others are comparatively more realistic. Moreover, in many cases, the end-users do not get the privilege to participate in the evaluation process.
1.2 Research Aim
The study aims to help end-user choose an effective open-source PMS using a structured and repeatable model where no intensive performance testing is involved. This paper aims at proposing a maturity model with the help of the current assessment models. The proposed model can motivate users to seek out the best available PMS that meets their requirements. In the proposed flexible assessment model, end-users, who want to manage their project, can select the desired candidate based on their priority. A set of pre-defined
criteria from this study will assist the evaluator in making their decision more efficiently and effortlessly. Although the research mainly will focus on PMS assessment, this study's output should help evaluate any categories of open-source software.
Our planned model makes the open-source PMS selection process easier by following a step-by-step process whenever they face a decision problem assessing an open-source PMS. This project should help the end-users to select their OSS with less technical knowledge. The study especially helps the evaluators realize the important criteria that motivate an end-user to choose an OSS. The most influenced criteria and sub-criteria derived from this study will guide the evaluators when assessing OSS projects.
1.3 Research Questions
The research question of this study is following:
How to adopt the existing open-source software assessment model for evaluating and selecting an open-source project management software?
The investigation considers the following four sub-questions for finding the answer:
1) Which criteria affect the OSS assessment?
2) What are metrics that measure the criteria?
3) How could these factors be evaluated?
4) Which criteria are most important for selecting a PMS?
The research examines the OSS phenomenon, licensing regulation, license compatibility, and current assessment models to answer the first three sub-questions. Literature review of the current assessment model helps the author propose an assessment model to answer the third sub-question. The information gained from the first two sub-questions aids in designing an online survey that will help answer the fourth sub-question.
1.4 Research Methods
The research includes a literature review and a survey. A large literature review helps to understand the OSS phenomenon, various types, license policy of OSS, the current OSS assessment models. Studying the current assessment models helps to find a set of metrics to measure each assessment criteria. The assessment process of the existing models also will be studied to understand their usability and flexibility. The knowledge of the current
open-source assessment model help to develop a new flexible open-source assessment model where evaluators or users need to collect data for each criterion and calculate the maturity score of their OSS candidates. Then, an online survey is conducted to determine the most influencing criteria for assessing an open-source PMS. According to the desired goal, evaluators will find their best choice of several alternatives based on the desired target.
The comprehensive research includes six chapters. The first chapter consists of research problem recognition, research aim, research questions, and methods. The relevant information presented in the second chapter includes basic OSS definitions, types of OSS, OSS license policies, OSS license compatibility. The four most popular and mature assessment models are studied in the third chapter to understand the assessment criteria of OSS to develop a new flexible open-source assessment model. The fourth chapter represents an online survey of end-users who has used or plans to use PMS to determine the most influential criteria for assessing open-source PMS. The survey's result is analyzed and discussed in chapter five. Finally, chapter six summarizes the research's significance and future work.
2 Open Source Software
As a foundation for the research, we will introduce the concept of OSS in this chapter.
This chapter covers the types of software, the definition of OSS, OSS license policy, and license compatibility of OSS.
2.1 Types of Software
Knowing the different types of software is vital for selecting the right software or making any software-related decisions. The Free Software Foundation (FSF) divided package software into a few categories, as shown in Figure 1. Proprietary software or non-free software has restrictions on use, delivery, or distribution imposed by the licensor.
Although all free software is part of OSS (indicated in Figure 1), OSS is not always free software due to differing license policies. Most free software uses General Public License (GPL) (Vainio & Vadén, 2012). Simultaneously, OSS can use the GPL or another license that allows for non-free software incorporation (Elliott & Scacchi, 2008). Section 2.2 presents a detailed discussion about the license policy.
Figure 1: Categories of software (Bagayoko et al., 2010)
2.1.1 Free Software
Free software does not always imply that it is free, but its policy respects users' freedom.
Free software allows users to inspect, modify, review, and distribute the program at any time. During the formation of the FSF in 1984, Richard developed the GNU software to alternative the Unix operating system with "free" source code for all users (Elliott &
Scacchi, 2008). In the middle of 1990, Richard stallman introduced Free software that follows the GNU GPL in the software development process (SCEPANOVIC et al., 2011).
Richard Stallman describes free software as liberty, not price, and set up the four essential freedoms principle. The four freedom criteria provide a clear distinction between free and non-free software. The software must meet the following four freedom criteria defined by Stallman to be called Free Software (Vainio & Vadén, 2012):
1. The freedom to use the software for any reason;
2. The freedom to research and modify it;
3. The freedom to copy it;
4. The freedom to develop it and make it public.
Freedom 1 ensures that users are not prohibited or prevented from running free software.
It has nothing to do with the program's functionality, whether it is theoretically capable of running in any given environment or valid for any specific computing task. For in- stance, if the code rejects or fails those meaningful inputs unconditionally, the program can become less usable, if not completely useless. However, it does not deny users the freedom to run the program, so it does not interfere with freedom 1. Users will resolve the loss of use if the software is open since freedoms 2 and 4 enable users and communi- ties to create and distribute updated versions without the arbitrary nuisance code (Vainio
& Vadén, 2012).
Accessing the program's source code is essential for freedoms 2 and 4. As a result, source code accessibility is a requirement for free software. Freedom 2 allows the users to ex- amine the source code and modify it before using it. Merging free subroutines and mod- ules is a helpful way to change a program. If the program's license requires any restrictive permit to modify the code, it can not be considered free. One example of freedom two is deleting the program's code to run another program. Therefore, the right to uninstall or delete the software is possible according to freedom 2 (Vainio & Vadén, 2012). Freedoms 3 and 4 allow anyone to redistribute anywhere, for free or for a fee, with or without changes. Users do not need to ask or pay for permission to share the program because of these freedoms. Anyone has the right to make modifications and use them privately in their work, without even mentioning that they exist. Any change to the original software and releasing it is permissible without asking anyone. Freedom 3 entails the ability to distribute the updated versions as OSS.
Copylefted Software
Software developers use the principle of "copyleft" to ensure unrestricted access.
Copyleft software uses copyright's legal mechanism to ensure the software and its source code remain available to everyone (Vainio & Vadén, 2012). The most important criterion is that distributing the software must be done under the original license (Morin et al., 2012). Copyleft licenses are labelled "restrictive" because of this provision, even though the limitations ensure free access. Copyleft is a type of copyright law that does the reverse of what it's supposed to do: instead of privatizing software, it keeps it free (Morin et al., 2012). Proprietary software developers take away users' rights by using the copyright;
Meanwhile, copyleft software ensures their (Androutsellis-Theotokis et al., 2010). As a result, the authority changed the name from "copyright" to "copyleft."
GPL'ed covered Software
GNU GPL-licensed software is that which has a GNU GPL license. Software released under GPL is a copylefted software (Morin et al., 2012).
Public domain Software
The source code of public domain software is free to modify or redistribute (Androutsellis-Theotokis et al., 2010). SQLite (SQLite.org, 2014) is a good example of public domain software. It is also legal to acquire public domain software and resell it under non-open license systems or delete the author's name and deal it as an independent work.
Xfree86 Style Software
Since 1992, the XFree86 Project, Inc has been a global volunteer group dedicated to pro- ducing XFree86 (XFree86® Home to the X Window System, 2014). It is free and OSS under the XFree86 license. The X Window System is implemented in XFree86. In a nut- shell, XFree86 is the most popular open-source X11-based desktop environment (XFree86® Home to the X Window System, 2014).
Open Source Software
"OSS characteristics allow anyone to use, edit and distribute for free, but it does not al- ways have to be cost-free (Tekale, 2015). " OSS development focuses on creating broad, accessible, and separated groups of developers who believe in software freedom and col- laborate by sharing knowledge and assisting others. The ability to view the source code
is not the only advantage of OSS. OSS policy also makes sure there is no discrimination during the distribution process. The following are some of the OSS characteristics and benefits abstracted from its definition:
• Free Distribution: There are no license costs associated with OSS (Androutsellis-Theotokis et al., 2010). The license does not stop anyone from sell- ing or distributing. For such a transaction, the license does not entail a royalty or other charge.
• No Discrimination: No discrimination is involved in the OSS development process. The license cannot discriminate against individuals or groups of individuals (Androutsellis-Theotokis et al., 2010). The license policy must not exclude someone from using the software in a particular area of endeavour. It does not, for example, prohibit the software from being used in a company or for genetic testing. The licensing must not exclude someone from using the software in a particular area of endeavour. It does not prohibit the software from being used in a company or for genetic testing. The primary goal is to avoid license traps from preventing the OSS from being used.
• Source Code Availability: The product package includes the source code. If a product does not have source code, there must be a well-documented way to access it for low cost, preferably via free download. The source code must be the preferred method of modifying the software by a programmer. It is not permitted to use intentionally confusing source code. Intermediate form such as interpreter is not allowed.
• Modifications and derivative works: Users of the OSS can alter the source code to make their derivative software (Androutsellis-Theotokis et al., 2010). However, depending on the OSS license used, this could be subject to additional restrictions.
Modifications and derivative works and their dissemination must follow the same licensing terms as the original program.
• Distribution of License: The program's privileges must be extended to everybody who receives it, without the requirement for additional authorization
from specific parties. This provision protects software from being shut down inadvertently, such as when a non-disclosure agreement is requested.
• License Must Be Technology-Neutral: No license requirement depends on a particular technology or interface style. This provision applies to licenses requiring a simple gesture of assent for the licensor and the licensee to form a contract. Click-wrap conditions can obstruct critical software distribution methods like FTP download, CD-ROM anthologies, and web mirroring and prohibit code.
• A License Must Be Product Independent: The program's privileges must not be conditional on the program's inclusion in specific software distribution. For example, assume that the software is derived from the distribution and used or distributed under the program's license terms. In this case, both parties to whom the application has been distributed should have the same privileges as those given by the original software distribution. This provision now prohibits another form of license trap.
2.1.2 Proprietary Software
This type of software publisher or another entity reserves certain rights from licensees to use, alter, distribute updates, or share the software. It may include patent privileges. From Figure 1, we came to know that closed and shareware software is part of proprietary software. Downloading proprietary software is possible, but source code will not be available. Even if proprietary software's source code is downloadable, users cannot change or redistribute it, separating it from OSS characteristics. Relicensing, selling, and modification of proprietary software are not allowed. The majority of software is proprietary and created by a third-party vendor.
Closed Software
The source code is not available to programmers in this type of software. The source code for this program is the intellectual property of the software publishers. Since only the original authors can copy, alter, and share the software, it is known as "proprietary software." Java (Oracle Corporation, 2021), Photoshop (Official Adobe Photoshop |
Photo & Design Software, n.d.), Microsoft Office (Office 365 Login | Microsoft Office, 2018) are the most common examples of closed software.
Shareware Software
Shareware is paid software but is made available for free for a limited time, known as a
"trial period." A trial period allows free usage, but the seller will ask users to buy it after that. Users will try out applications before purchasing the paid version. We are familiar with shareware software like Adobe Photoshop (Official Adobe Photoshop | Photo &
Design Software, n.d.) and Adobe Illustrator (Adobe Inc., 2012).
2.1.3 Comparison between OSS and proprietary software
As shown in Figure 1, both OSS and proprietary software are available for free download.
In contrast to proprietary software, OSS attracts more developers worldwide than proprietary software because of the source code availability and the freedom of modification. However, it is not opposed to commercial software. Table 1 gives an overview of the significant differences between OSS and proprietary software.
OSS Proprietary Software
Users have access to the source code Purchased without source code Users are allowed to make changes to the
software.
Modification is not possible The software can be freely installed on
any computer by users.
Before installing the software, users must first obtain a license from the suppliers.
Support is not always available Easier access to support No guarantee for continuous develop-
ment
Consistent feature development Table 1: The difference between OSS and proprietary software
2.2 License Policies of OSS
Users have the right to access, copy, develop, and distribute software under open source licenses. These rights and conditions are present in the software's license, a contract between the licensors and the licensees (Androutsellis-Theotokis et al., 2010). OSS licenses come in a variety of flavours. However, they all make software source code
accessible and allow for the development of derivative works and non-exclusive commercial use of the original and derivatives. The OSS's licensor is a single or a group of developers or even an organization that owns the software's copyright. The licensor decides what OSS license to use for releasing their work. The licensee is either an end- user of OSS or a company that has integrated it into a product.
Although software licenses have gained media coverage due to the Free Software Foundation's efforts, scholars have paid little attention to licensing evolution, despite its many potential adverse effects on software reuse (Di Penta et al., 2010). The GPL is the most commonly used copyleft software license, with fair terms of use. A user can change software under the GPL license without the permission of the author. At the same time, the BSD license places limitations on software alteration without the developer's permission. If the user plan to use software that is not under the GPL license policy, it is essential to make sure the license does not contain any provisions that are not compatible.
Figure 2 indicates that OSS's License mainly consists of copyleft and permissive license.
Figure 2: A categorization of OSS license types, their main properties, and some characteristic examples (Androutsellis-Theotokis et al., 2010)
2.3 License Compatibility
Users must determine if their licenses are compatible or not if they merge two OSS into one or merge code from one into another. There is no issue combining programs with the same licenses, but we must check their compatibility for a different license. In general, licenses are compatible if it is possible to combine code under several licenses. Figure 3 presents a summary of license compatibility. The vector arrows in Figure 3 indicate one- way compatibility, implying that compatibility on the left ("permissive licenses") is more vital than compatibility on the right ("copyleft licenses"). From Figure 3, we can see BSD- new is one of the most compatible licenses. If the user's software is under the BSD-new license, it can combine with other software with any LGPL versions, Apache, or MPL license. For example, if the user's license is X and combines with another software with license Y, the resulting software will be under license Y.
Figure 3: License compatibility diagram (Nyman, 2015)
2.4 Summary
OSS is not always free due to different license policies, but the source code is always available in the OSS project. OSS's openness and freedom attract more developers than proprietary software. During the OSS assessment process, it is essential to check the license policy and its compatibility. Combining two OSS projects is possible if both share the same license, but users must consider license compatibility for a different license type.
3 Open Source Software Assessment Models
The chapter focuses on the current OSS assessment models to determine the essential criteria and their metrics for the assessment. The knowledge of the current assessment model helps to develop a new flexible open-source assessment model at the end of this chapter.
There are several models available for assessing OSS. These models focus on various aspects such as durability, maturity of OSS (Lenarduzzi et al., 2020). Some consider functionality in their model, whereas others concentrate only on the quality of OSS. OSS projects are generally assessed based on recommendations and previous experiences. This research will look at the following four most commonly used assessment models to see how they assess OSS:
• Capgemini Open Source Maturity Model (C-OSMM) (Duijnhouwer & Widdows, 2003)
• Navica Open Source Maturity Model (N-OSMM) (Russo et al., 2010)
• Open Business Readiness Rating (Open BRR) (Standard et al., 2005)
• Easiest Open Source Selection (E-OSS) (Houaich & Belaissaoui, 2015)
The first three models were selected considering their seniority (Umm-E-Laila et al., 2017). We selected the E-OSS model because it is simple, easy to perform, and well- descriptive (Houaich & Belaissaoui, 2015). We will focus on two essential things for all these models: firstly, criteria and subcriteria, secondly, their evaluation process.
3.1 C-OSMM
Capgemini developed the C-OSMM model back in 2003. The goal of developing this model was to assist businesses in determining the best OSS for their requirements based on the program's age (Adewumi et al., 2019). It reaches the goal into account both product and application indicators. Application indicators are consumer expectations and future needs, while product indicators are factual and observable information about the product (Umm-E-Laila et al., 2017).
3.1.1 Product indicators
Product indicators use twelve weighted parameters or criteria for the evaluation process to be entered into a spreadsheet and analyzed for the desired solution (Ahmad & Laplante, 2011). Product, integration, use, and acceptance are the four groups separated by the
twelve criteria (Duijnhouwer & Widdows, 2003). The product group reflects the open- source project's internal consistency or quality. Measuring the capability of OSS for grouping with others is the main concern of the Integration group. The use group focuses on the standard of the OSS project and the service facility to its users. The acceptance category evaluates how the general public receives the product (Russo et al., 2010).
Product Group
Figure 4a shows that the product category includes five essential metrics: age, selling points, developer community, human hierarchies, and licensing (Adewumi et al., 2015).
The age of OSS refers to how long it has been operating. The older the program is, the less likely it is to be abandoned unexpectedly. A business trust a product more when they know that the product will be supported regularly by the developers. Selling point refers to the features not available in existing software and corrects a significant flaw in that software or features of significantly higher quality than the existing software.
The metric, developer community, is another essential component for keeping the OSS alive. The more people involved in software development, the more likely it is to continue evolving and improving. It's challenging to develop a project that a single individual ultimately manages. Creation, code review, testing, and support are all easier to monitor with hierarchical management. For successful software creation, hierarchical management is a wise option. Finally, licensing is the most crucial factor when choosing an OSS, as a very restrictive license can limit the potential usage or distribution of the derived work.
Figure 4a: Product indicators of C-OSMM model (Duijnhouwer & Widdows, 2003)
Integration Group
The integration category has two metrics: modularity and collaboration with other products, as shown in Figure 4b. Modularity makes the OSS project more beneficial for the developers. A well-developed OSS can have many independent functional modules responsible for specific tasks. Modularity comes in handy where some people may not want to use the whole project but few essential components or modules based on their need. Modularity saves the time and effort of the developers. A high-quality software should be easy to integrate with other software. The willingness of an OSS to collaborate with other products determines the number of users.
Figure 4b: Product indicators of C-OSMM model (Duijnhouwer & Widdows, 2003)
Use Group
Figure 4b indicates that the Use group mainly consists of two essential metrics: standard and support. Software standard refers to a common standard that software developers embrace and use when operating on one or more computer programs. An OSS having the most commonly used standard makes it more adopted by the community. Support can refer to the vendor's one-on-one help to technicians and end-users concerning hardware, operating systems, and software. Highly accessible support plays a vital role in choosing and using an OSS project. When users have issues with the applications or request new functionality, they must assist the developers and user groups.
Acceptance Group
From Figure 4c, we can see that the acceptance group includes ease of deployment, user community, and market penetration. Proper documentation related to deployment activities such as release, installation, activation, deactivation, uninstallation, update, version tracking is significant. Such documentation works as a user manual for the new users of an OSS project. It saves new users a lot of time looking for information and helps them get on the right track faster. User community refers to the active user of an OSS. An extensive and active community demonstrates the value of an OSS and aids in obtaining new users. Developers are more interested in contributing to a large community project.
Market penetration indicates the percentage of consumers who use a product relative to the estimated demand. Therefore, a higher percentage of market share indicates that an OSS is more mature (Duijnhouwer & Widdows, 2003).
Figure 4c: Product indicators of C-OSMM model (Duijnhouwer & Widdows, 2003)
3.1.2 Application indicators
Application indicators deal with the user's expectations and future needs of an OSS project (Umm-E-Laila et al., 2017). C-OSMM model is more than a maturity model as it also takes the user's requirement criteria into account. The user measures whether the software is suitable or not with the help of the application indicators. The application indicators include fifteen indicators, as shown in Figure 5.
Figure 5: Application indicators of C-OSMM model (Duijnhouwer & Widdows, 2003)
Usability is a metric that evaluates how simple a user interface is to use. When evaluating it, we should look at how simple it is for users to complete fundamental activities for the first time and how enjoyable it is to use. Interfacing indicator examines how interactable is the software for the user (Duijnhouwer & Widdows, 2003). Performance indicators can measure how successfully a software system achieves its goals. The response time is the amount of time it takes to respond to a request, which is essential for measuring software performance.
The reliability indicator depends on the availability of the software, where the availability of software is the probability that a system is operational at a particular moment. The software availability indicator depends on the likelihood that a system will be operational at any given time. In addition, the software must not fail during its execution. Software is reliable if it does not crash during runtime and functions correctly under certain conditions. A security indicator is a metric that indicates how safe a system is from danger and risk. There are various methods and metrics for measuring security. For instance, we can measure the security of a system by counting the number of days between the vulnerability and when it is corrected.
The vendor independence indicator is the most influential criterion that attracts OSS users. It indicates the level of commitment between client and supplier for using a product (Duijnhouwer & Widdows, 2003). Platform independence indicator refers to an application's ability to run across a variety of operating systems. It increases the product's value and needs. For example, product X is compatible with Windows and Linux, and product Y is only compatible with Linux. In that case, product X will be given top priority in the selection process. Support indicators are mostly related to the importance of the system. It refers to the quality and availability of professional support. Reports enable tracking and sharing the scope, time, budget, and status of a project to everyone who needs to know at any time. It aids in keeping a project on top and manage the expectations of end-users.
The administration indicator determines whether or not the product is compatible with existing maintenance tools (Duijnhouwer & Widdows, 2003). It consists of operation management. Advice means the validation or recommendation by independent parties for using a particular product. The number of feedback, recommendations of end-users and expert users plays a significant role in product selection. Training in implementing very modular software is essential for many end-users. Many OSS projects provide required training to make the project more acceptable to the audience. Training includes web-based mini-tutorials created by developers, commercial tutorials, classrooms organized by the developers. Staffing refers to the continuous process of hiring people and training them to improve a project and make it more competitive. The implementation criterion handles the process of adopting and integrating a software application into a company workflow.
3.1.3 C-OSMM Assessment Process
The C-OSMM model consists of seven steps in order, as shown in Figure 6. In the first step, select alternative products from various sources such as OSS hosting platforms, the internet, suggestion from OSS developers and friends. Then, a score ranges from one to 5 is given for each product indicator. In the third step, the application indicators are scored collaborating with the customer (Russo et al., 2010). In step four, a priority value is determined based on the customer's interview to demonstrate the importance of various application indicators. Each application indicator's priority value ranges from one to five, representing unimportant and extremely (Duijnhouwer & Widdows, 2003). Earlier from Figure 5, we have noticed how the final score of the application indicators is calculated.
After multiplying the priority (P) with the score (S), the metrics are given their final values (Duijnhouwer & Widdows, 2003). The weight factor will be five if the indicator 'performance' has a priority of five and the OSS project has a score of one. The score is three if an indicator is not related to the evaluation procedure (Russo et al., 2010).
Figure 6: C-OSMM process (Russo et al., 2010)
3.2 N-OSMM
Novica's CEO, Bernard Golden, created the N-OSMM (Navica Open Source Maturity Model) in 2004 (Umm-E-Laila et al., 2017). Golden created this model to assist organizations in evaluating OSS and determining whether a product would meet their needs. This model has 3 phases (Umm-E-Laila et al., 2017). Phase 1 is to identify and evaluate critical OSS maturity criteria. The next phase is to define the weighting factor for each metric defined in phase one in phase 2. Finally, phase 3 adds the weighted metrics scores to get the final maturity ranking (Umm-E-Laila et al., 2017). As compared to C- OSMM, N-OSMM has fewer assessment criteria. It is, however, well-structured, with three main phases that ensure consistency and usability.
Phase 1
Six elements are the critical metrics in phase 1: product software, support, documentation, training, product integration, and professional service (Tawileh et al., 2006). The majority of the N-OSMM criteria are identical to the C-OSMM model. The most weighted criterion of this model is the product software consisting of several sub-criteria such as product functionality, longevity. Product functionality sub-criteria is the essential criteria comparing others. It is the first thing that drives a user to adopt the product. If the product functionality does not fulfil the minimum usage criteria of the user, then there is no possibility of using that product. Longevity refers to the life expectancy of OSS projects.
Because commercial companies do not manage OSS projects, their longevity depends mainly on community initiatives.
The support criterion is the second most influential criterion for the OSS Selection process. The availability of community support or paid support is crucial when a user faces a particular problem or requesting a new function. Sub-criteria such as web postings, developer-created documentation, and commercially published documentation assist in measuring the documentation criteria. Successful documentation makes information easily available, aids new users in quickly understanding the product, simplifies the product, and lowers support costs.
Training is an essential criterion to show the user how the software works. The need for training after software adoption is critical for achieving optimal efficiency and lowering associate turnover. Following the implementation of the software system, there are numerous options for delivering training materials to the users. The developer may provide training by creating a commercial tutorial or classroom lesson. OSS users can obtain training from existing web-based mini-tutorials for a particular product.
Product integration criterion is responsible for measuring the ability of a particular OSS project with other projects. Professional services such as technical support, consultation, and training are crucial for keeping an OSS project alive and competitive. Many OSS companies are providing professional services to their customer for a fee. In addition, the product team of an OSS project or external firm may provide professional services to the users.
Phase 1 consists of a four-step procedure (Russo et al., 2010). The consumer determines their specifications and describes the essential criteria for their needs in the first stage.
The second step is to find available resources, either inside the company or through the OSS community. The third most important step is assessing the maturity of each element.
Assign a maturity score to each criterion is the last step in phase 1. The score ranges from zero to ten, indicating how well the element meets the consumer's needs (Russo et al., 2010).
Phase 2 & Phase 3
Phase 2 give each of the vital criteria a weight based on its significance. Naturally, not all aspects of a product are equally essential. For example, the software is essential;
assistance is crucial; while required, documentation is secondary to the first two. As shown in Figure 7, the N-OSMM model comes with a default weighting factor that is changeable to meet the needs of each organization. Phase 3 combines the scores for vital criteria and their weighting factors to calculate the product maturity score. The maximum value for the multiplication of the assigned element score and weighting score is 100.
Figure 7: Default OSMM element weightings (Navica - Open Source Maturity Model (OSMM), 2018)
3.3 Open BRR
In 2005, a flexible and scientific assessment model known as the Open BRR (Business Readiness Rating) model was developed (Umm-E-Laila et al., 2017). The goal of creating this model is to help IT managers Figure out whether OSS is the best fit for their needs (Adewumi et al., 2019). The Open BRR approach goes a step further than the previous approaches in that it suggests an open and versatile evaluation that is standardized at the same time (Russo et al., 2010). As a result, it enables a more comprehensive implementation and a more straightforward evaluation of open-source and closed-source applications. Furthermore, unlike other versions, the Open BRR has five grades, ranging from 1 to 5, where one refers to "Unacceptable", and five represents "Excellent." As shown in Figure 8, The entire evaluation method consists of four phases (Standard et al., 2005).
Figure 8: The four phases of the Open BRR model (Standard et al., 2005)
Phase 1
Phase 1, the quick assessment filter, aims to exclude products that do not meet the basic requirements (Adewumi et al., 2019). Users can filter products based on various qualitative and quantitive characteristics (Russo et al., 2010). The characteristics can include the product's legality, standards, implementation language, and others (Standard et al., 2005). These characteristics or filtering criteria do not take away the user's freedom.
Users can add filters based on their needs in this model.
Phase 2
In Phase 2, it is critical to consider which categories and metrics to use for the in-depth evaluation processes (Umm-E-Laila et al., 2017). The Open BRR model divides into twelve categories, each with its own set of metrics. Figure 9 shows the twelve categories
proposed for this model. These categories are fairly similar to the ones used in N-OSMM and C-OSMM. Users must choose seven or fewer additional important categories and assign percentage weights to each that total 100%. Users rate metrics within each category and give a percentage weighting factor that adds up to 100% based on their significance.
A metric that helps measure OSS criteria can be quantitive or qualitative, but both need to be normalized (ElFadl et al., 2018). Compared to the qualitative category, the quantitative category is comparatively simple to normalize (Standard et al., 2005).
Figure 9: The criteia of Open BRR model (Standard et al., 2005)
Phase 3 and 4
In Phase 3, the user collects the necessary data for each metric, which must then be compared to a normalized scale to make the data useful. This phase is time consuming compared to others. If we look at the "number of downloads," this phase should show whether 1000 to 5000 is an "appropriate" range for measuring maturity. This process, however, may include binary metrics. The last phase, data translation, calculates the numeric maturity score. As shown in Figures 10a and 10b, the model provides commonly used metrics and their scoring.
Figure 10a: Metrics and scoring of the Open BRR model (Standard et al., 2005)
Figure 10b: Metrics and scoring of the Open BRR model (Standard et al., 2005)
The functionality category measures how well the product meets user requirements. If product functionality does not match basic requirements, it will not pass the first phase of this model. As explained in the C-OSMM model, usability criteria indicate the easiness of handling a particular software. However, this model provides more metrics with a transparent scoring system for calculating usability. The Open BRR model considers end- user UI experience, setup time for installing OSS, and time for vanilla installation or configuration time when calculating usability. Quality category aids in determining how complete and error-free a software is. It focuses on software's design quality, code quality and test quality. The number of releases in the last twelve months, number of open and closed bugs, and age of critical bugs are the metrics that help calculate the software's quality (Samoladas I, Gousios G, Spinellis D, 2008).
The security category refers to the software's level of security and its ability to handle security issues. The availability of security information is useful when calculating the security score. The performance category measures how well does a system perform under certain conditions. This model focuses on the availability of performance data and performance testing to score the performance category. Unfortunately, some OSS projects only provide the best performance factors, not a complete performance scenario (Wheeler, 2011). This unanticipated occurrence also occurs with proprietary applications.
Mailing to OSS projects, on the other hand, can assist in gaining a clear picture of the performance situation.
The scalability of software refers to the amount of data or problems it can handle. When selecting an OSS, it is essential to ensure that software has been evaluated on large datasets or run on massively parallel or distributed computers. Documentation regarding the scalability and a real-world deployment test can help to measure the scalability. The architecture category measures the modularity, portability, flexibility and extensibility of an OSS. This model checks whether the software allows an extension through a public API key or any third-party plugin to calculate architecture category score. As noticed and discussed earlier, support is one of the most common and influential categories for selecting an OSS. The availability of a mailing list or forum is the first thing to check for support. In addition, many mature OSS projects may provide paid professional support for software installation, troubleshooting and integration.
The significance and need of the documentation category are explained in the last model.
The Open BRR model considers various documentation and feedback from existing users to measure the documentation criteria. The documentation about installation, deployment and upgrade should be available in different formats such as PDF, HTML. The adoption category defines how well do community, market, and industry adopt the component. The number of books available about a component and its scalability details helps to measure its adoption score. The community category assesses how involved and vibrant is the community for a particular OSS software. The community category is scored based on the number of unique contributors to an open-source project and the size of the mailing list. The professionalism category manages the product quality by creating a barrier to entering the development process. An immature project often has no option not to allow new committers in the development process. However, a mature OSS project must be selective about new committers to maintain the code quality.
3.4 E-OSS
SIAD-laboratory developed E-OSS (Easiest Open Source Selection) model in 2015 (Umm-E-Laila et al., 2017). Figure 11 depicts the four phases of the E-OSS model:
definition, identification, qualification, and selection. The primary purpose of this model was to promote free software in Morocco (Houaich & Belaissaoui, 2015). The E-OSS model combines the previous approaches with the associated criterion "interoperability,"
enabling adopting a new open-source product capable of integrating with existing products (Houaich & Belaissaoui, 2015).
Figure 11. Four stages of the E-OSS model (Houaich & Belaissaoui, 2015)
Definition Phase
The first phase, the definition phase, is a critical stage that identifies a company's actual requirements when adopting a new OSS. Checking and analyzing the following aspects
is critical for a better summary and knowledge gathering: "functional, technological, and strategic" (Houaich & Belaissaoui, 2015). The functional component outlines the essential business functions such as accounting and sales, allowing a user to choose only the applications that will perform as anticipated (Umm-E-Laila et al., 2017). The technical component refers to the secondary aspects that can impact a product's success (Houaich
& Belaissaoui, 2015). It is essential to consider the people who are using the current system. Knowing their expectations from the new system will help a user avoid switching to a new one. Finally, the strategy component refers to a company's strategy. For example, some businesses may have exclusive software contracts with a proprietary software vendor that is a subsidiary of a holding company and only needs specialized software (Houaich & Belaissaoui, 2015).
Identification Phase
The identification phase aims to decide the OSS's general characteristics to construct a data sheet describing all possible products (Houaich & Belaissaoui, 2015). This model uses twelve indicators: seniority, licensing, human hierarchies, developer community, performance, feedback, interoperability, security, functionality, training, support, and documentation to calculate the product score. The E-OSS model divides these twelve indicators into four categories: product, quality, integration, and facility, as shown in Figure 12. This flexible model always allows users to add new criteria based on their needs. All the indicators of the product group are similar to the product group of the C- OSMM model.
Figure 12: Different categories of identification phase of E-OSS phase (Umm-E-Laila et al., 2017)
Qualification & Selection Phase
The qualification phase aims to assign a score to criteria and get an overall score to select the best product (Houaich & Belaissaoui, 2015). In this model, each indicator can have a score ranging from 0 (unacceptable) to 5 (very convincing) (Houaich & Belaissaoui, 2015). Thus, utilizing all of the characteristics discussed in the two sections,
"Identification & Qualification," it is possible to select the most developed OSS to fulfil a variety of requirements (Houaich & Belaissaoui, 2015). The last step calculates the total number of all criterion scores to classify the various mature solutions (Umm-E-Laila et al., 2017).
Product Group
The product group consist of seniority, licensing, human hierarchies and developer community criteria. These criteria are precisely the same as the C-OSMM product group;
only the selling point criterion is missing here. We have already investigated the importance of age or seniority criterion in the C-OSMM model. Mature product is more reliable, and company feel assurance of regular improvement by its developers. Figure 13 indicates the measures of the product group. If a product's age is less than a year, then the score for seniority criterion is zero, but if age is more than nine years, then the score is maximum which is five. When selecting an OSS, it's essential to be cautious and double-check the license that comes with it. The BSD license, for example, allows for source code changes and a license modification for newly changed code to make it private. On the other hand, the GPL license does not allow the newly updated code license to be changed.
Continuous development and improvement of an OSS largely depend on the number of developers involved. Many programmers worldwide work on OSS around the clock (Tekale, 2015). The involvement of many developers in a project makes the development process faster and makes the project more innovative. From Figure 13, we can see the E- OSS model provides a scale for scoring the developer community criterion. The human hierarchies criterion refers to the responsible person for managing a project. One person can not run a project for the long term. Therefore, a project with many people has more opportunities for improvement in the production and service sector.
Figure 13: Metrics of the product group (Houaich & Belaissaoui, 2015)
Quality Group
Performance and feedback criteria are both included in the quality category. The model provides the measurement scale of the quality group, as shown in Figure 14. The free software performance depends on the service provided, such as response time when accessing data from the database. The model considers the minimum score of zero if the response time exceeds six seconds and the maximum score of five if the response time is less than fifty milliseconds. Feedback of the current users plays a significant role when selecting an OSS. Feedback is essential for determining the benefits and drawbacks of a product without even installing it. This model considers the number of posts in the OSS forum to calculate the feedback criterion score. Figure 14 shows that if the product has less than 5000 posts, a score of 0 is assigned, and if it has more than 8000 posts, then a score of 5 is assigned.
Figure 14: Metrics of the quality group (Houaich & Belaissaoui, 2015)
Integration Group
As explained in the C-OSMM model, the integration focuses on OSS's ability to group with other projects. The integration category includes requirements for interoperability, security, and functionality. The ability of software to interact with one or more specific systems is known as interoperability. Interoperability was scored based on the product compatibility with the technologies deployed, as shown in Figure 15. Functionality criterion deals with the user's requirement. According to the model calculation scale for functionality requirements, if the product does not have some necessary features, it receives 0. if the product has any required features, it receives a score of 3, and if the product has all required features, it receives a score of 5. Finally, the number of significant risks of the last 12 months is the measures of security criterion. The lower the number of risks, the higher the score of the security.
Figure 15: Metrics of the integration group (Houaich & Belaissaoui, 2015)
As Figure 16 indicates, the facility group includes training, support and documentation criteria. For a business, it is essential to train people and update their competencies before implementing a new OSS to replace an existing one. As a result, when choosing an OSS software, inquiring about training availability is critical. In addition, there should be two types of documentation: first, the user documentation that explains software usage and the second, developer documentation that guides the developers for any modification in source code. According to this model, the score of documentation criteria is zero if the product has no documentation at all. If documentation exists but is not updated, then the
score is three. However, the presence of updated documentation scored as five in the E- OSS model.
Facility Group
Any system's existence depends on its ability to provide support. It is a critical element that offers strategies for ensuring the smooth operation of business processes and output continuity. There are two types of services available: free and charged. In most instances, community members provide free assistance for OSS. The service areas in the community are vital tools for resolving issues. If a product has paid support, as shown in Figure 16, the score for the support criterion is five. The score is three if the product only has community interest. If none of these forms of support is available, the score will be zero.
Since users are the ones who are in charge of installing and maintaining OSS, accurate and up-to-date documentation is essential.
Figure 16: Metrics of the facility group (Houaich & Belaissaoui, 2015)
3.5 Comparison of Assessment Models
The research focuses on four assessment models of OSS. Figure 17 indicates a side by side comparison of these models. As shown in Figure 17, the comparison factors used for each model are their assessment criteria, scoring scale, target audience, flexibility, repetitiveness and how easy it is to perform the overall assessment process. Comparing all these factors helps to measure the adaptability of each model. Comparison of these models also helps determine how they perform their overall assessment process and the criteria used for the assessment. Except for Open BRR, all models are practical and follow properly outlined techniques (Umm-E-Laila et al., 2017).
Figure 17: Comparison of OSS maturity models (Umm-E-Laila et al., 2017)
The number of criteria used in each of these models varies, as shown in Figure 18.
Comparing to all other models, the C-OSMM model has the most criteria, which is twenty-seven. An extensive set of criteria helps select a product with higher quality, but dealing with too many criteria demands a handful of time. Measuring a large set of indicators becomes complex when OSS information is limited. Only the E-OSS model includes interoperability criteria to make the OSS suitable for enterprise-level development. The interoperability criterion is not present in any of the models except E- OSS. That is one of the requirements for both Commercial software and OSS to be called enterprise suitable.
All these methods mentioned in Figure 17 use the Weight Score Method (WSM) to calculate the evaluation process's final result except C-OSMM. Unfortunately, the final score computation technique is not defined in the C-OSMM model (Umm-E-Laila et al., 2017). The C-OSMM model's shortcomings include the lack of tool support for evaluating OSS alternatives and its lack of scientific validation in the literature (Adewumi et al., 2013). Moreover, the C-OSMM model did not explain precise quality measurements for assessing the quality characteristics (ElFadl et al., 2018), while the N- OSMM, Open BRR, and the E-OSS model provides the measurement scale in detail.
C-OSMM, Open BRR, and E-OSS model use a scoring scale from one to five to score all the criteria. However, N-OSMM is too open, allowing a one to ten scoring scale, often leading to extra complexity and hesitation during the scoring process. Except for Open BRR, none of these models considers the normalization policy while scoring. In the Open BRR model, users need to normalize the collected data about the metrics, which is
incredibly challenging and time-consuming. The Open BRR model's target audience is more extensive than any other model as it deals with large organizations, small and medium businesses and even private users. The Open BRR model has no tool support (Adewumi et al., 2013). The E-OSS model is the most recent compared to others, and it is the easiest to perform because of its simple measurement scale.
Figure 18: Criteria of C-OSMM, N-OSMM, Open BRR, and E-OSS model
3.6 Model Development 3.6.1 Assessment Criteria
The model presents a set of important criteria for assessing OSS, as shown in Figure 18.
Consumers can select top n criteria while assessing a list of OSS. The proposed model included the interoperability criterion from the E-OSS model missing in the other three assessment models essential to select appropriate OSS for enterprise-level. A fundamental criterion, documentation, was included in the suggested model, missing in the C-OSMM model. The N-OSMM and E-OSS model missing criteria such as reliability, market penetration, standards, scalability, and platform independence included in the proposed model. Open BRR does not include training criteria, which undoubtedly play a significant role in adopting an OSS. It is also missing essential criteria such as human hierarchies, market penetration, modularity, feedback. All these missing criteria of the Open BRR model are included in the proposed model.
3.6.2 Assessment Process
Figure 19 indicates the seven steps involved in the proposed assessment model. The overall assessment process of the proposed model is explained below:
Step 1: The consumers set their requirements that an OSS must fulfil.
Step 2: Find alternative software from available resources such as open-source hosting platforms.
Step 3: Consumer selects top n criteria from Figure 18 according to their need.
Step 4: The consumer needs to assign a weight to all criteria for each product. The overall weight of all the criteria needs to be 100.
Step 5: Consumers assign a maturity score to all the criteria of each product. The score ranges from one to five, indicating how well the software meets the consumer's needs.
Step 6: Consumers multiply each criterion's weight and maturity score and calculate the overall score of each using a weight scoring model.
Step 7: The consumer finds their desired software that has the maximum score.
