Software Evolution and Feedback: Theory and Practice - Hardcover

Hardcover

ISBN 10: 0470871806 ISBN 13: 9780470871805

Publisher: Wiley, 2006

Synopsis

Evolution of software has long been recognized as one of the most problematic and challenging areas in the field of software engineering, as evidenced by the high, often up to 60-80%, life-cycle costs attributed to this activity over the life of a software system.  Studies of software evolution are central to the understanding and practice of software development.  Yet it has received relatively little attention in the field of software engineering.  This book focuses on topics aimed at giving a scientific insight into the aspect of software evolution and feedback.

In summary, the book covers conceptual, phenomenological, empirical, technological and theoretical aspects of the field of software evolution - with contributions from the leading experts.

This book delivers an up-to-date scientific understanding of what software evolution is, to show why it is inevitable for real world applications, and it demonstrates the role of feedback in software development and maintenance.  The book also addresses some of the phenomenological and technological underpinnings and includes rules and guidelines for increased software evolvability and, in general, sustainability of the evolution process.

Software Evolution and Feedback provides a long overdue, scientific focus on software evolution and the role of feedback in the software process, making this the indispensable guide for all software practitioners, researchers and managers in the software industry.

About this title

About the Author

Nazim H. Madhavji and Juan Fernandez-Ramil are the authors of Software Evolution and Feedback: Theory and Practice, published by Wiley.

From the Back Cover

Society is becoming increasingly dependant on software at all levels of human activity. At the same time, software needs to be kept up-to-date, that is, evolved, if it is to satisfy its purpose in a changing world. However, evolution of software has long been recognized as one of the most problematic, challenging and expensive areas in the field of software engineering. Software Evolution and Feedback is a comprehensive reference to concepts, phenomena, and techniques to assist the maintenance, management and understanding of very large and long-lived software systems. The book provides an up-to-date scientific overview of what software evolution is, shows why it is inevitable for real-world applications, how to technically and managerially deal with it, and demonstrates the role of feedback in software development and maintenance.

Read on to find out more about:

• the theoretical underpinnings and conceptual understanding of software evolution,
• how requirements change over time due to external factors,
• the characteristics of proprietary and open-source software evolution,
• the relationship between software architectures and software evolution,
• the evolution of object-oriented frameworks,
• feedback and system dynamics issues in the software process,
• the use of policies to guide software evolution,
• the role of metrics that characterise the risk of making changes … and much more.

Software Evolution and Feedback will provide practitioners and managers in the software industry, as well as academics, graduate students and researchers in the area of software engineering, with sound scientific insight into software evolution and feedback.

Read on to find out more about:

• the theoretical underpinnings and conceptual understanding of software evolution,
• how requirements change over time due to external factors,
• the characteristics of proprietary and open-source software evolution,
• the relationship between software architectures and software evolution,
• the evolution of object-oriented frameworks,
• feedback and system dynamics issues in the software process,
• the use of policies to guide software evolution,
• the role of metrics that characterise the risk of making changes … and much more.

Software Evolution and Feedback will provide practitioners and managers in the software industry, as well as academics, graduate students and researchers in the area of software engineering, with sound scientific insight into software evolution and feedback.

Excerpt. © Reprinted by permission. All rights reserved.

Software Evolution and Feedback

Theory and Practice

John Wiley & Sons

Copyright © 2006 John Wiley & Sons, Ltd
All right reserved.
ISBN: 978-0-470-87180-5

Chapter One

Software Evolution

Meir Lehman and Juan C. Fernndez-Ramil

This chapter is a revised version of the paper by Lehman MM and Ramil JF, Software Evolution and Software Evolution Processes, Annals of Software Engineering, special issue on Software Process-based Software Engineering, vol. 14, 2002, pp. 275-309, with kind permission of Springer Science and Business Media.

1.1 Introduction

1.1.1 Evolution

Evolution describes a phenomenon that is widespread across many domains. Natural species, societies, cities, concepts, theories, ideas all evolve over time, each in its own context. The term reflects a process of progressive, for example beneficial, change in the attributes of the evolving entity or that of one or more of its constituent elements. What is accepted as progressive must be determined in each context.

It is also appropriate to apply the term evolution when long-term change trends are beneficial even though isolated or short sequences of changes may appear degenerative. Thus it may be regarded as the antithesis of decay. For example, an entity or a collection of entities may be said to be evolving if their value or fitness is increasing over time; Individually or collectively they are becoming more meaningful, more complete or more adapted to a changing environment.

In most situations, evolution results from concurrent changes in several, even many, of the properties of the evolving entity or collection of entities. Individual changes are generally small relative to the entity as a whole, but even then their impact may be significant. In areas such as software, many allegedly independent changes may be implemented in parallel. As changes occur as a part of the overall evolution, properties no longer appropriate may be removed or may disappear and new properties may emerge.

The evolution phenomena as observed in different domains vary widely. To distinguish between domains, one may start by classifying them according to their most evident characteristics. A study of common factors shared by subsets of their entities, distinctions between them and their individual evolutionary patterns may suggest specific relationships between evolution and other properties and indicate how individual patterns and trends are driven, directed and even controlled.

One could, perhaps, increase understanding of software evolution by studying instances of the phenomenon in other domains. The discussion here is, however, limited to the computing and software fields.

1.1.2 Interpretation of the Term Evolution in the Context of Software

The term evolution in the context of software may be interpreted in two distinct ways, discussed more fully in Chapter 16 [Lehman and Ramil 2001b]. The most widespread view is that the important evolution issues in software engineering are those that concern the means whereby it may be directed, implemented and controlled. Matters deserving attention and the investment of resources relate to methods, tools and activities whereby software and the systems it controls may be implemented from conception to realisation and usage, and then evolved to adapt it to changing operational environments. One is seeking continuing satisfactory execution with maximum confidence in the results at minimum cost and delay in a changing world.

Means include mechanisms and tools whereby evolution may be achieved according to plan in a systematic and controlled manner. The focus of this approach, termed the verbal approach, is on the how of software evolution. Work addressing these issues has been widely presented and discussed, for example, at a series of meetings titled Principles of Software Evolution (e.g. IWPSE 2004).

An alternative approach may also be taken. This less common, but equally, important view seeks an understanding of the nature of the evolution phenomenon, what drives it, its impact, and so on. It is a nounal view investigating the what and why of evolution. Far fewer investigators (e.g. Lehman et al. 1969-2002, Chong Hok Yuen 1981, Kemerer and Slaughter 1999, Antn and Potts 2001, Nanda and Madhavji 2002, Capiluppi et al. 2004) have adopted it. It is driven by the realisation that more insight into and better understanding of the evolution phenomenon must lead to improved methods and tools for its planning, management and implementation. It will, for example, help identify areas in which research effort is most likely to yield significant benefit. The need for understanding and its significance will become clearer when the nature of, at least, the industrial software evolution process as a multi-loop, multi-level, multi-agent feedback system (Lehman 1994) is appreciated. Failure to fully appreciate that fact and its consequences can result in unexpected, even anti-intuitive responses when software is executed and used.

There is a view that the term evolution should be restricted to software change (e.g. Mittermeir 2006). However, under this interpretation, important activities such as defect fixing, functional extension and restructuring would be implicitly excluded. Other authors have interpreted evolution as a stage in the operational lifetime of a software system, intermediate between initial implementation and a stage called servicing (Bennett and Rajlich 2000, Rajlich and Bennett 2000). These and still other interpretations are covered by the areas of evolution presented below. They are, therefore, not separately identified in the present chapter.

1.2 The Evolution of Large Software Systems

1.2.1 Early Work

As stated in Lehman's first law of software evolution, (Lehman 1974), it is now generally accepted (e.g. Bennett and Rajlich 2000, Pfleeger 2001, Cook et al. 2006) that E-type software must be continually adapted and changed if it is to remain satisfactory in use. Universal experience that software requires continual maintenance (as evolution was then termed) was first publicly discussed at the Garmisch Conference (Naur and Randell 1968) and viewed as a matter of serious concern.

At about that time, Lehman reported on his study of the IBM programming process (Lehman 1969), though his report did not become generally available till much later (Lehman and Belady 1985). Inter alia, the report examined and modelled the continuing change process being applied to IBM's OS360-370 operating system. Preliminary models of that system's evolution were derived from measures and models of release properties. Refined versions of these were subsequently proposed as tools for planning, management and control of sequences of releases (Belady and Lehman 1972, Lehman 1974, 1980).

Recognition of the software process as a feedback system brought the realisation that the study of the process and its evolution must consider that fact, if more effective management and process improvement was to be achieved. This observation triggered an investigation of the phenomenon initially termed Program Growth Dynamics (Belady and Lehman 1972) and later Program Evolution Dynamics (Lehman 1974). The resultant study produced not only fundamental insights into the nature and properties of the software process but also into those of its products. Early studies concentrated on OS360-370 release data; later studies involved other systems (Lehman 1980, Lehman and Belady 1985). All in all, the results of these studies greatly increased understanding of the software evolution phenomenon and identified practices and tools for its support (Lehman 1980).

1.2.2 Large Programs

Lehman and Belady's early work on software growth dynamics and evolution concluded that evolution is intrinsic to large programs. This adjective has been variously interpreted as applying to programs ranging in size from 50 and 500 thousand of lines of code (Klocs). Subsequently, Lehman suggested that such an arbitrary boundary was not very useful in the evolution context. It appeared highly unlikely that one could identify, even approximately, a single bound over a spectrum of programs such that those on either side of the divide displayed different properties. Moreover, if size were a major factor in determining evolutionary properties, one would expect these to change for programs of different size.

Moreover, it was seen as unlikely that all would appear at around the same loc level, independently of, for example, application, organisational, managerial, process and computational factors. Any of these might relate to the emergence of disciplined evolutionary behaviour. As a result of considerations such as these, Lehman suggested that the observed phenomena were more likely to be linked to properties related to characteristics of software development, usage and application environments and processes or of their products. He, therefore proposed that a program should be termed large if '... it had been developed or maintained in a management structure involving at least two groups' (Lehman 1979), that is, subject to, at least, two levels of direct management. This property appeared sufficient to explain many of the observed evolution dynamics properties of the systems studied.

This definition followed from the recognition of the fact that development by an individual or small group subject to the direct control of a single individual, is quite different to one in which there are two or more management levels. When a single manager is in day-to-day control the focus of goals and activities will be a matter of ongoing discussion and decision within the group, subject to a final approval by the manager. With two or more groups and managers at two or more levels of management, each level, each manager, each group will develop its individual goals, understanding, language, interpretations etc. Communication between the members of any individual group will tend to be continual and informal. Between groups and levels it will tend to be discontinuous and more formal. This will cause divergence of the terminologies, technologies, interpretations, goals, and so on as perceived and applied in and by the separate groups. Such divergence is clearly a major source of the 'large program problem' (Brooks 1975) and that problem, in turn, appears to be one of the drivers of software evolution. It must also be recognised that in cooperative multi-group activity, it is human nature for individual groups and their managers to seek to optimise their own immediate results, overlooking or ignoring the impact on other groups and on the overall, long-term consequence.

Furthermore, programs developed by the joint effort of multiple groups are functionally rich and structurally complex. Their effective development and use requires the application and integration of many skills and approaches and communication between participants. It was thought at one time (Lehman 1979) that the resultant activities will favour the emergence of evolutionary characteristics associated with programs that have traditionally been termed large. This further supported the above definition of largeness. However, the latter was still considered unsatisfying as a complete explanation for the intrinsic need for software evolution.

1.3 Program Classification

1.3.1 The SPE Program Classification Schema

Despite the revised definition, the concept of largeness appeared unsatisfactory as the fundamental basis for a study of software evolution. To address these concerns, a program classification scheme, not involving a concept of size was proposed. Initially, this defined programs of types S, P and E (Lehman 1980, 1982, Pfleeger 2001) as discussed below. The third is the most closely related to a discussion of software evolution. Though not a defining property of the phenomenon, it has been shown as inevitable for that class of program if users are to remain satisfied with the results of its use. Subsequently, it was realised that the classification is equally relevant to computer applications, application domains, application and computing systems and so on (Lehman 1991).

1.3.2 S-type Applications and Software

1.3.2.1 Definition

A program is defined as being of type S if it can be shown that it satisfies the necessary and sufficient condition that it is correct in the full mathematical sense relative to a pre-stated formal specification (Lehman 1980, 1982). Thus a demonstration, by means of proof for example, that it satisfies the specification (Hoare 1969, 1971), suffices for contractual completion and program acceptance. Where it is possible, for example with the exception of systems where decidability issues arise (Apt and Kozen 1986), demonstration of correctness is also a matter of mathematical skill and the availability of appropriate tools. The proof demonstrates that program properties satisfy the specification in its entirety. Such verification suffices for program acceptability if the specification is satisfactory to intended users and meets their requirements. That is, the specification will have been validated and accepted. Completion of verification then justifies contractual acceptance of the program.

The definition assumes implicitly that a specification can be predetermined before development begins and that, once fixed, learning during the course of the subsequent process is restricted to determination of methods of solution and the choice of a best method (in the context of constraints applying in the solution domain). Implementation is driven purely by the implementers' knowledge, understanding and experience.

The designation S was applied to S-type systems to indicate the role played by the specification in determining product properties.

1.3.2.2 Validation

The above implies that verification with respect to the specification completes the S-type development process. If satisfaction of the specification by the final program product is (contractually) accepted as sufficient by both developer and client, verification leads directly to acceptance. Practical application of the S-type development process, however, requires that the specification is valid in the context of its intented use. Validation of a specification is, in general, nontrivial.

1.3.2.3 The S-type in a Changing Domain

Even if initially satisfactory, changes in the use of an S-type program or in its operational environments or circumstances can cause it to become unsatisfactory. In this event, the specification, the problem or both must be revised. By definition, this means that a new program based on a new specification is being implemented. However, the new derivation is likely to be based on previous versions of the specification and program, that is, the latter are modified rather than recreated. Conceptually, however, evolution of S-type programs is restricted to the initial development. It consists of a discrete sequence of processes each of which includes specification revision, program derivation and verification.

1.3.2.4 Formal Specification

Application of the S-type concept is limited to formally specifiable problems. It also requires that a procedure for computation of the solution is known or can be developed within budgetary and time constraints. In other words, there are four conditions that the S-type program must satisfy in order to be legitimately termed as such. First, the problem can be rigorously, that is, formally, stated. Second, the problem must be solvable algorithmically. Third, it must be feasible to prove that the program is correct with respect to the formal specification. Last, but not least, the specification must be complete, that is, final for the moment (see Section 1.3.2.3), in terms of the stakeholders' current requirements. It must explicitly state all functional and nonfunctional requirements of concern to the stakeholders and, in particular, clients and users. Nonfunctional requirements include the range and precision of variables, maximum storage space, execution time limits, and so on.

(Continues...)

Excerpted from Software Evolution and Feedback Copyright © 2006 by John Wiley & Sons, Ltd. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Edition details

• Publisher: Wiley
• Publication date: 2006
• Language: English
• ISBN 10: 0470871806
• ISBN 13: 9780470871805
• Binding: Hardcover
• Edition number: 1
• Number of pages: 612
• Editor: Fernandez-Ramil Juan, Perry Dewayne, Madhavji Nazim H.