Product Software Quality.pdf

TECHNISCHE UNIVERSITEIT EINDHOVEN

Department of Mathematics and Computing Science

MASTER’S THESIS

Product Software Quality

Dipl.ing. Gregor Panovski

Graduation supervisor:

Dr. Alexander Serebrenik (LaQuSo, TU/e)

Graduation committee members:

Dr. Natalia Sidorova (TU/e)

Dr. Marko van Eekelen – (LaQuSo, RU Nijmegen)

Graduation tutor:

Ir. Petra Heck - LaQuSo (TU/e)

Eindhoven, February 2008

Preface

The MSc. study of “Computer Science and Engineering” at the Eindhoven University

of Technology (TU/e) is concluded with master thesis project. This research was

conducted at the LaQuSo (Laboratory for Software Quality) in Eindhoven.

In the first place, I would like to thank Eindhoven University of Technology and

LaQuSo for giving me the opportunity to graduate in such interesting domain. I would

like to thank Petra Heck and Alexander Serebrenik for their supervision, advices,

comments on the thesis, pleasant cooperation and feedback. I thank to Marko van

Eekelen for his practical advices and participation in graduation committee, Natalia

Sidorova for participation in graduation committee, Jovan Pehcevski and Jasen

Markovski for their readings and comments on my thesis, and all other employees of

LaQuSo, for the wonderful time and nice working environment at LaQuSo.

Finally yet importantly, I would like to thank my family, especially to my daughter

Mia and my wife Natasha and to my parents Ljiljana and Vladimir for their, support

and encouragements.

Abstract

Product Software is a commercial software package, containing a software application

and accompanying documentation. Software Quality is defined as conformance of the

produced software to stated and implied needs [ISO/IEC 9126-1]. In order to

understand and measure quality, scientists often built models of how quality

characteristics relate to each other. A quality model is a set of quality characteristics

and relationships between them, which provides the basis for evaluating product

software quality and specifying its requirements [ISO/IEC 9126-1].

Evaluation of product software quality is the topic of this M.Sc. project at the

Eindhoven University of Technology conducted at the Laboratory for Quality

Software (LaQuSo). The project is focused on evaluation of external quality , which

means assessing the behavior of product software when it is executed. In this thesis,

we present our experiences and guidelines for evaluating quality of product software

applications from different application domains.

The major research question we addressed is how one should evaluate external

product software quality. As quality is known to be a complex and multidimensional,

subject to many constraints, it follows that quality evaluation is a complex process as

well. Hence, to fully address this question, the following related sub-questions are

considered:

•

Is product software quality domain dependent?

•

How can we use the ISO/IEC 9126-1 quality model in different domains?

•

What are the differences between product software quality and tailor-made

software quality?

Current quality models such as ISO/IEC 9126 contain numerous metrics and their full

usage requires significant evaluation effort per product. Accordingly, we focus on a

subset of metrics that are relevant for chosen application domains and to evaluate

quality with the relevant metrics only. We also conduct a survey contacting the

software producers in the Netherlands asking them which ISO/IEC sub-characteristics

are important for their product software. We analyzed the results, but the response

was not sufficiently high to perform a relevant statistical analysis. We believe that the

industrial response was limited due to marginal use of the ISO/IEC 9126 standard in

the industry.

As a starting point in the quality evaluation, we divided the software products in three

categories: infrastructure software , software development tools, and application

software [OECD]. Further, we executed product analysis in order to define which

quality sub-characteristics are relevant for specific and related products classified in

the same category. The reason for this was to reduce the evaluation effort focusing on

relevant characteristics per category only. To create category specific quality models,

we departed from the ISO/IEC 9126 standard and made use of the methodology

proposed in [Botella] for building ISO/IEC 9126-based quality models. The

methodology consists of several steps and the basic idea is to derive domain specific

metrics starting from ISO/IEC 9126 quality characteristics. In our work, we

decomposed each of the relevant ISO 9126 sub-characteristics in more concrete

entities, called attributes and proposed metrics for these attributes. As a guideline for

metric definition, we used the external metrics provided by [ISO/IEC 9126-2]. The

first part of the metrics was literary taken from [ISO/IEC 9126-2], the second part of

the metrics was inspired by the [ISO/IEC 9126-2] metrics and derived for the specific

product, while the third part were metrics not defined by the standard but related to

the application domain of product software.

Using the above methodology, we analyzed seven product software examples from

the three listed categories and completely evaluated four product software

examples: two examples of application software and two software development tools.

Different ISO 9126 characteristics were relevant for the three product software

categories. For example, functionality is very important for all the three categories but

portability is not important for any of them. For the other three ISO 9126

characteristics (usability, reliability and efficiency), we have discovered significant

differences between product software belonging to different categories. Usability is

very important for application software products, but it is less important for software

development tools and infrastructure software. Reliability and efficiency, on the other

hand, are very important for infrastructure software but less important for the other

two categories. With this analysis, we prepared reduced quality models per category.

Using these reduced quality models on our evaluated product software examples we

reduced software evaluation time to one week per product software, compared to the

time of more than one month per product when using full quality models.

We focused on the metrics provided by [ISO/IEC 9126-2]. We found that several

[ISO/IEC 9126-2] metrics require presence of internal information and documents

such as requirements specification or the number of faults during development. These

documents may not be available for external evaluation, as they contain company

confidential information. Hence, we expect this subset of [ISO/IEC 9126-2] metrics to

be used for assessment by an internal evaluator only. Other [ISO/IEC 9126-2] metrics,

such as the efficiency metric Throughput and security metric Access controllability ,

are too general, so the evaluator should refine or translate them to metrics specific for

the product. Finally, the third group of [ISO/IEC 9126-2] metrics can be widely

applied in different domains.

We conclude that external product software quality is domain or category dependent.

We created reduced quality models based on ISO 9126 that can be applied per product

software category. Using these reduced quality models, we decreased the evaluation

effort per product software. As a proof of concept, we evaluated the external quality

of four product software applications from different domains.

Metrics provided by [ISO/IEC 9126-2] can be used as a starting point for metrics

definition, but in our opinion, they are not “ready to use”. Thus, the evaluator should

adapt them to the product category or to the domain and business objectives.

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika: