Applying Non-Functional Requirements Analysis to an Existing Project – An Experience Report

Abstract

The importance of non-functional or quality requirements in system development is now widely recognized. While a number of approaches have been proposed to treat quality attributes as properties of a completed design or a finished product, the NFR framework treats non-functional requirements as goals which guide the generation and selection of design choices during the development process. As a comprehensive approach to supporting the progressive refinement of requirements and the development of design solutions, the framework was meant to be used from start to finish in a project. In this paper we consider the use of the NFR framework on a smaller scale, for the analysis of the architectural design of a system in an existing project. Although the project did not use the framework to support its development, we found that an opportunistic application of the framework to a small, selected portion of the system could still offer substantial insight to the development team, by illustrating how important high-level quality requirements - including business objectives and product goals - are impacted by specific design decisions. The positive responses in this study lead us to believe that a scaled-down version of the NFR framework might be useful on its own, despite much reduced capabilities, in more restricted contexts, such as architectural inspection or review, and also as an initial demonstration of concept to gain acceptance for a larger scale adoption of the framework.

1  Introduction (1p)

Non-functional requirements have been regarded as an essential part of a requirements document since the early days of software engineering [Boehm?, Thayer dorfman? ROME?]. While functional requirements define what a software system should do in relation to its environment,

non-functional requirements define qualities that the system should have, such as its usability, reliability, performance, security, accuracy, costs, maintainability, evolvability, etc. The success of a software system often depends on how well these qualities are achieved.

Much of Requirements Engineering (RE) has focused on dealing with functional requirements. Many techniques have been developed to specify the functional characteristics of software systems, usually in terms of inputs and outputs, states and transitions. Formal notations and languages have been developed to model and specify systems and environments. These techniques aim to overcome the many limitations of requirements expressed in natural language or in informal diagrams, such as ambiguity, imprecision, inconsistencies, incompleteness, poor traceability, maintainability, and sheer volume. Many of these approaches offer powerful analysis techniques to assist verification and validation, such as the checking or proving of mathematically formulated properties, and the simulation or animation of behaviours. There are also well developed methods and guidelines for eliciting functional requirements. [Davis? Wieringa? Sommerville Kontoya?]

Non-functional requirements have turned out to be a much greater challenge. Although the importance of non-functional requirements is widely acknowledged in RE and in software engineering. there has been much fewer advances compared to the treatment of functional requirements. A major step was the explicit identification and classification of non-functional requirements (also called quality attributes, or “-ilities”) [Boehm79?] [RADC]. Much work has been done to deal with particular areas of software system properties, such as reliability, security, usability, and so forth. Advances in these areas have often included formal and sometimes quantitative techniques. However, a major challenge is in how to deal with the interactions among many competing (and sometimes complementary) non-functional requirements from many areas. It could be argued that understanding these interactions, and making the appropriate tradeoffs is one of the most difficult tasks and valuable contribution of the seasoned architect or designer. Yet, there are few systematic approaches or frameworks to guide requirements engineers and system designers in dealing with non-functional requirements in a comprehensive way. While the incompatibility of techniques from different areas poses one problem, any problem is that many techniques are aimed at analyzing and evaluating a system only when the system has been developed, or at least when there is a concrete design. These methods offer little help during requirements engineering, and even during architectural design. If non-functional

requirements are as crucial to system success as we profess them to be, then we should aim to develop techniques to overcome ambiguity, imprecision, inconsistency, incompleteness, and so on as early in the development cycle as possible, just as there are techniques for functional requirements.

The NFR framework [Chung 93] [CNYM98] was developed to address…((this vacuum! This holy grail that has eluded the RE community so far. J How do you like this, Lawrence? ))

…has been proposed as a process-oriented, goal-oriented approach to addressing software quality [CNYM98]. The approach puts system-wide quality requirements (also known as non-functional requirement—NFRs) up-front as goals to be achieved. This is especially important during high-level component design, i.e., in making architectural decisions [CNY95].

During the architectural design process, goals are decomposed, alternatives are analyzed with respect to their tradeoffs, design decisions are made and rationalized, and goal achievement is evaluated. This way, system-wide properties serve to systematically guide selection among architectural design alternatives.

-  systematic refinement helps disambiguate and incompleteness of coverage (inc. thru use of parms

-  supports tradeoffs through correlation detection and analysis.

-  precision by incremental decomposition

-  conciseness through graphical representation

-  traceability through links in the graph.

-  revisability through evaln procedure supported by truth-maint. mech.

The framework was originally conceived with the aim of providing a comprehensive approach to development from initial requirements to detailed design. It would be used from the start of a project. It would also cover the whole breadth of the project. While this deployment setting would be the ideal, and in order to obtain the full benefit of the framework, it also presents considerable obstacles.

-  requires substantial commitment from the project orgn., and management

-  changes to way of thinking, support tools, …

-  acceptance by individuals

So, even though the idea may be appealing and convincing in concept, it faces hurdles in getting an organization to adopt it, even on a pilot trial basis.

One approach ((to overcome the initial hurdles)) that could be taken is to apply the framework in an adapted setting, a limited way, on an existing system, in a much reduced scope, with limited commitment, -- but which would nevertheless give the potential users a strong feeling about the potential benefits (and possible limitations) of the approach. The compromises include

-  analyze existing system instead of from scratch

-  small scope, little commitment required, little effort (min. interference because they have little time to consider new techniques)

-  (requires adaptations on our side)

purpose:

-  to obtain buy-in, as well as initial motivation

-  to get feedback on how to tailor to this organization.

-  to understand their needs, to see whether, and what features are of special interest to them.

Analyze a system that is being developed. Can directly and personally relate to the issues raised by the exercise. Can feel the power and also limitations. what they could personally gain, and how the overall project might be affected.

In this paper, we report on our experience in taking this approach in a study conducted with an industrial firm. In a short period, we carried out a NFR analysis on a software system that was being developed at this firm. The analysis was primarily through the extracting information from a key document. We codified relevant information according to the NFR framework concepts and notation. (( and performed some simply analysis)). We presented the results to project members who were directly involved in the project. The responses were extremely positive, both from a management group and from a developer group. This exercise resulted in the company’s decision to select another (new) project ((about to be launched)) to pilot test the proposed approach ((NFR + i*)) in a more comprehensive way.

In section 2, we describe the background of the study.

In section 3, we briefly review the main technical concepts of the NFR framework and the adaptations that were needed for the limited application setting.

In section 4, we explain the step we took in order to carry out the analysis. This is illustrated with a (slightly simplified) version of the main example we used in the presentations.

Section 5 presents the reactions we received from project participants, as well as what we learned from the experience.

Section 6 relates our experiences to related work, and discusses the implications.

Section 7 presents final conclusions and outlines future work.

1.1 

2  Background ((Case study context ( ¼ p) ))

-  brief overview of the system, the development team, the organization.

The study site was the software development organization of a medium-sized telecommunications products manufacturer with world-wide offices and sales. More than many other industries, telecommunications has become intensely software driven. The ability to develop new software quickly and effectively is essential for success in the business. Massive and radical changes in the industry … deregulation, open technology standards, have led to industry restructuring, ((companies repositioning themselves and their products in a rapidly changing customer and competitive environment)), … intense competition, accelerating time pressure.

Business goals have changed … time to market is paramount.

Reliability still a distinguishing market leader.

But open interface standards,

evolvability. painfully aware from previous generation products.

Juggling these key issues are important to the product managers, who are responsible for product success. They need to identify and define products and communicate to system developers.

However, difficult for technical designers to appreciate the practical implications on their day-to-day work, e.g., design decisions.

The R&D group recognized these are in fact high-level non-functional requirements – time-to-market, usability, reliabililty, etc., which need to be propagated downwards to guide specific design decisions. The R&D group initiated a research project with the university research team, with funding support from a government agency. After initial discussions, the study described in this paper was carried out.

The system was a new product launched a few months before the study was undertaken. The architectural design phase was completed. The research team was given access to the project documentation.

3  Analyzing Non-Functional Requirements

3.1  The NFR framework - a brief summary ( ½ p)

-  brief recap of main features of NFR. No figures.

-  (( for this paper, maybe we wont have time or space to include i*. We will mention i* on the side.))

The NFR Framework [Chung93] [CNYM98] has been proposed as a process-oriented approach to addressing software quality [CNYM98] in general, complementary to the traditional product-oriented approaches such as software metrics. The approach puts system-wide quality requirements (also known as non-functional requirements—NFRs, and “-ilities”) up-front as goals to be achieved, hence being goal-oriented.

This is especially important during high-level component design, i.e., in making architectural decisions [CNY95], since the space of architectural decision space is potentially infinite. Instead of merely evaluating several fully-specified architectures only at the end of the architectural design process, the Framework’s goal-oriented approach encourages the exploration of partially-specified alternatives at numerous decision points throughout the process and use of system-wide properties to select among such alternatives.

During the architectural design process, goals are decomposed and prioritized, alternatives are explored and analyzed with respect to their tradeoffs, design decisions are made and rationalized by way of arguments, and goal achievement is evaluated in consideration of the characteristics of the application domain. More specifically, the Framework represents NFRs as softgoals (based on a ‘good enough’, satisficing approach to qualitative reasoning), and relationships between softgoals as contribution types (both partial and full, as well as “+” or “-“). These softgoals can be in conflict or synergy with each other. Each softgoal is associated with a satisficing status indicating the degree to which it is satisficed or denied, or in conflict. All the essential ontological concepts of NFRs and their refinements are diagrammatically represented in a softgoal interdependency graph (SIG).

The NFR framework provides further systematic support for the software development process with:

Ø  generic methods: a body of knowledge of softgoal satisficing,

Ø  correlation rules: a body of design tradeoffs, and a

Ø  graph labeling procedure for propagating satisficing status.

A SIG typically begins with a number of abstract NFRs, which are incrementally refined with the help of generic methods, correlation rules and the labeling procedure, until an effective solution is found. These solutions are called operationalizations, which meet functional and non-functional requirements.

As we will explain later, we did not apply the NFR framework as it was intended, in a top-down manner. ***the previous sentence does not lead to following*** ???As a result??? we found it necessary to simultaneously consider various levels of architectural decisions. This meant moving from [NFRs] to operationalizations, and from operationalizations to a new set of [NFRs]. ***** There is no provision in the framework, for attaching a new set of [NFRs] [as subgoals] to an operationalization. To remedy this problem we used a hybrid of the NFR Framework and i* framework [Yu95] which offers an agent-oriented approach to modelling organizations through a set of agent ontology including agents, task and resources.

[describe I* and hybrid]

4  Illustration of the Analysis ( 3p, inc. 3 – 4 figs.)

-  (( one example from top level NFR’s down to design decisions, tradeoffs ))

-  OAMP, Sys. Mgt Forms.

-  (( condensed from WP Part 1, sec. 3 ))

4.2.  Steps in the Process

Initial survey.

survey of the documents. orientation to the application domain, context. ((step –2))

Nature of the Document. We worked with a System Level Architecture Document [SLAD]. The document is complete and well organized by current professional standards. It begins with an introduction, which states the purpose and scope of the product, lists reference documents, and gives a history of changes to the SLAD. A section on requirements and a separate section on constraints follow. These three sections comprise six pages. The next forty pages describe the software architecture. The final forty pages are appendices. They include an architectural analysis and detailed discussions of the applicability of standards.