STEP 3: DETERMINE CURRENT PRODUCT QUALITY 3-23
Acquire product quality data Activity
3-1
• Prepare product data acquisition
• Acquire product quality data
• Report product quality data
The purpose is to collect data from the product regarding the product characteristics (e.g. ISO 9126), with the help of product measurement and/or a product characterization questionnaire. This data will be used to identify which product areas are already satisfactory, and which ones require additional work. The questionnaire examines the product’s important quality characteristics. PROFES does not provide generic questionnaires for this activity. We recommend that one is made for every product type being developed. The results of Step 2 can be used to set up such a questionnaire.
Existing measurement data can be also used here. Parts of the necessary product information may be achieved using data from earlier measurement programmes, measurement of similar products or older version of the product. Product information can also be derived from verification and validation activities performed during past development cycles.
In Step 2, an survey of product quality needs has been made in which product quality needs were specified, translated into ISO9126 terms, and enhanced with measurable criteria. These criteria were specified in the form of metrics and the required value was specified as well. In this step, we suggest that data be collected for these metrics and a check made whether the current value for these metrics is in line with the required value specified in Step 2.
Measuring the data for these product quality metrics can be done by the quality engineer who manages the project. However, it may be necessary to involve a member of the project team as well. Specific metrics may possibly not be calculated yet, due to the current development status of the product. Estimates by the project team are sufficient, as long as they do not know the required value before estimating it. Project team members have a tendency to align their estimates with targets, which should be taken into account.
Before carrying out a product evaluation, proper preparation is necessary.
This involves inviting people, assembling equipment, installing testing environment, setting up product evaluation procedures, etc. Preparation of
STEP 3: DETERMINE CURRENT PRODUCT QUALITY 3-25
product quality data acquisition is the most important and time-consuming part of the work. Once all the pre-requisites are arranged, data collection becomes quite easy. The findings should be reported in a product quality data report, which reports the current values to the metrics and requirements set in Step 2. An example is provided in Table 3.1.
Table 3.1. Example product quality data
Number Wish Quality Question/Metric Target Current
84 The product has
to be usable
Usability Do users find the UI clear? Yes Yes
87 No usability
problems should occur
Usability # bugs with the status: Open and severity: Usability (as recorded in Defect tracking tool)
Max.
15
35
Evaluate current status of product quality Activity
3-2
• Analyse product quality data
• Discuss product evaluation results
• Determine current product quality
Based on the values of the questionnaire or the values that were measured during Activity 3-1, the current status of product quality can be determined. The result should be documented according to the ISO9126 quality (sub)-characteristics. This overview of current product quality status must indicate:
• What is the wanted product quality?
• What is the actual product quality?
• What is the difference there between wanted and actual quality? Is the quality excessive?
These differences are of especial interest, because they indicate which product quality areas the process is not sufficiently effective. This informa-tion can be very helpful in focusing the process assessment in Step 4. It is obvious that the areas in which improvement is needed are where the actual product quality is not acceptable. Therefore the output of Step 3 is also be highly relevant for Step 5, in which the product improvement goals will be set. It is also important to identify which product areas have a higher quality level than necessary.
The results of such an evaluation can be demonstrated by a diagram similar to that in Figure 3.3.
D C B A Functionality
Reliability
Usability
Efficiency Maintainability
Portability
Target Value Current Value
Figure 3.3. Example result of evaluating product quality status Based on such figures, it becomes possible to determine in which product quality areas improvements are required. In cases where the current product quality is lower than the target, improvements are particularly required.
Average Duration and Effort
The total duration and effort for the identification of current product quality will be about two person-weeks, depending on the specific product and the degree of difficulty in collecting the required data.
Tools and Templates
Product evaluation and measurement tools can provide suitable support for the step. MetriFlame can provide support during the analysis and representation of the measurement results. Other software tools can be used to calculate specific metrics on the software code, for example the number of source lines, or cyclomatic complexity.
STEP 3: DETERMINE CURRENT PRODUCT QUALITY 3-27
Work Products
Input work products Output work products
• Application domain characteristics
• Measurement data
• ISO9126
• Product quality profile
• Experience base
• Current status of product quality
Resource Allocation
Roles and responsibilities Managerial roles
Management
Management should be consulted on decisions for product quality. If not possible, management will at least receive the outcome of these decisions.
Marketing manager
Decisions on product quality levels are mainly taken in close co-operation with the marketing department. A marketing manager should be consulted during the decision process of what the product should or should not do.
Project manager
The project manager should be consulted during the activities to establish the current product quality. This effort will remain limited.
Expert roles PROFES expert
The PROFES expert carries out most of the activities in Step 3, and is responsible for product evaluation and determination of
the current status. The PROFES expert produces the deliver-ables from this step.
Support roles Software Engineer
It might be necessary to consult one or more software engi-neers during product evaluation of the current product. The software engineer will support the quality engineer where necessary. However, the role of the software engineer will be limited.
Expected effort/role PROFES expert
The effort spent by the PROFES expert on the above activities will be about 24 hours for the product data acquisition, depend-ing on the difficulty of settdepend-ing up the testdepend-ing environment, and 16 hours on the evaluation of current product quality and corres-ponding reporting tasks.
Project manager
The project manager will spend less than four hours on this step.
Software Engineer
The software engineer will spend less than eight hours on this step.
Marketing manager
The marketing manager will spend less than four hours on this step.
Management
Management will spend less than four hours on this step.
Methods and Techniques
No particular methods and techniques are provided for this step. Product quality metrics are measured, analysed, and comparing to the requirements set in Step 2.
1. VERIFY COMMITMENT
4. DETERMINE CURRENT PROCESS CAPABILITY 3. DETERMINE CURRENT PRODUCT QUALITY 2. IDENTIFY PRODUCT QUALITY NEEDS
10. IMPLEMENT AND MONITOR IMPROVEMENTS 7 . DESCRIBE PROCESS CHANGES
8. SET METRICS FOR THE PROCESSES AND PRODUCT
11. EVALUATE RESULTS
12 . UPDATE EXPERIENCE BASE
5. SET PRODUCT IMPROVEMENT GOALS
6. DETERMINE NECESSARY PROCESS CHANGES
9. PREPARE IMPROVEMENT IMPLEMENTATION CHARACTERIZE
PLAN
PACKAGE ANALYSE EXECUTE SET GOALS
PROFES PHASES PROFES STEPS
STEP 4: DETERMINE CURRENT PROCESS CAPABILITY 3-31