Incremental/Iterative Design
(These notes will be incrementally/iteratively improved upon.)
The Unified Process is a published framework from which incremental/iterative processes can be derived. It’s related to the Rational Unified Process.
The phases of incremental/iterative development can be catalogued in the Unified Process as:
Inception,… elaboration,… construction and… transition.
Construction is the main phase in which to iterate (shown by the vertical lines in the diagram above) but you can do so in any phase, particularly if it’s large.
Before we go on to the details of these phases, let’s take a side trip for a while. We’ll compare and contrast iterative/incremental and waterfall in too crudely simple terms, just to get the big picture clear. The main phase above is construction, just as waterfall really boils down to the stages analysis, design, coding and testing. Thus, at its most reduced and oversimplified, iterative development in a one year project would mean that you’d divide the project up into, say, four of three month iterations, each producing production quality software. Each iteration would consist of analysis, design, construction and testing. By comparison, if the year were spent doing the project using a waterfall type method, one might spend, say, two months doing analysis, four months doing design, three months construction and three months testing. Yet a third method of working would be to use staged delivery. That would entail, say, doing four months of analysis and design followed by four construction iterations, each producing production quality software.
Here’s a summary of the above oversimplification:Spending 12 months on a project:
Three alternatives:
Iterative/Incremental: 4 * 3 month construction increments each with analysis, design, coding and teting.
Waterfall: 2 months analysis, 4 months design, 3 months construction, 3 months testing
Staged Delivery (a hybrid of the above two) 4 months analysis and design, 4 * 2 month construction iterations, each including testing and each producing production quality software
So much for the oversimplification, now to more details of what an iterative/incremental project is like in reality when fleshed out. (Beware, however, that when many people talk about doing an iterative/incremental project, in reality they’re doing staged delivery!). We’ll still only be scratching the surface, however, so more reading, as well as practice, would make things clearer than what’s about to follow. A good book for supplementary reading is Craig Larman’s UML and Patterns. (see www.amazon.co.uk for more details, such as publisher, ISBN and reviews.)
Inception
Inception is largely about doing a few hours work to see if it is worth doing a few weeks’ work, or a few weeks’ work to see if it’s worth doing a few years’ work. For a very simple project, it may be a chat with the project sponsor over coffee; for a large project it might be a full-scale feasibility study.
Inception involves establishing:
(a) the domain-related rationale for the project (e.g. how much money it will bring it compared to costs, or perhaps if the system is less money-driven what benefits there will expected in, say, hospital patient care or reduction in accidents) and
(b) the scope of the project (i.e. the boundaries set on what aspects of the domain the system will tackle and what it won’t tackle). This can be very hard to decide.
At the end of the inception phase you have a go/no go decision by the project’s sponsors. Perhaps between 10 and 20% of use cases are worked out during this phase. Most of the remainder will be done during elaboration, but some of them might not be discovered until construction. Don’t expect to work out all the use cases before construction.
Discovering use cases:
One of the ways to discover use cases is to think of the actors with hats on (i.e. roles) that will be played in regard to the system. In addition one can consider the events that the system must respond to. It is not too worthwhile to consider actors, except insofar as they help to lead to use cases. Later, however, one might want to consider them again in order to tailor a use case to meet the needs of different actors, or to resolve priority issues that arise among competing needs, wants and, let’s face it, political power in an organisation.
The inception phase should never be omitted but it can be lightweight. It must be sufficiently intense to be sure that the project sponsor can make a sound go/no go decision.
Elaboration
First: Risk-driven- and client-driven iterative planning:
The UP encourages a combination of risk-driven and client-driven iterative planning. This means that early iterations are chosen to
(a) identify and drive down the highest risks
(b) build features that the client wants most.
Risk-driven iterative development includes the practice of architecture-centric iterative development, meaning that early iterations focus on building, testing and stabilising the core architecture. This is because not having a solid architecture is a common high risk. Thus we have architecture-centric, component-based design.
And now elaboration:
The elaboration phase, in particular, should be driven by a consideration of risk factors. Bringing risk up front in the project cycle and dealing with successfully is one of the hall-marks of iterative development. One has four principal sorts of risks to consider:
· Requirements risk
· Technological risk
· Skills risk
· Political risk
It’s in relation to requirements risk that UML techniques can most obviously be brought to bear. Use cases allow user requirements to be captured in manageable chunks. Class diagrams allow a rigorous vocabulary to be established for a system. Activity diagrams are useful, particularly, in the sort of system that has a clear work-flow element. In the early stages of working with the domain people, the developer resorts to these three models. The domain representatives are taught a bit of UML, if they don’t know any already, so that they can understand the diagrams that the developers are using. If however some UML is difficult for them to follow, then the developer ought to modify it accordingly by judiciously breaking some of the rules. The domain people may have among them software developers and analysts who work for them full time and they’ll help to serve as a bridge between domain people and developers. This would be particularly true in the case of a large multi-national or a government department.
When a development team is in the elaboration phase, it sits down with the domain people and listens to the words they are using such as customer, order, account and so forth. The developers become absorbed into the world that the domain people inhabit. The developers and domain people together tie these nouns together using class diagrams drawn from a conceptual point-of-view. The class diagrams help discover use cases and conversely, when a use case is discovered, the class diagram has to be looked at to see if there are missing elements. This means that there’s a to and fro movement in the mind between use cases and the class diagram, as shown in the diagrams above and below.
The vocabulary used in the verbal description of the Use Cases, both the main success scenarios and the alternatives that may exist at various points in the MSSs, are carefully mined to extract all the nouns that could help to inspire the objects on the conceptual class diagram (There will be a separate study of this during the academic session.)
It’s a good idea, in the initial stages, when in meetings with the domain representatives, to draw lots of diagrams informally and to write down as many accompanying notes around it as possible. Later on the conceptual model can be consolidated with the help of two or three people of the domain who like to get deeper into modelling and in whose heads there is a conceptual framework of how the organisation works. It’s worth remembering, incidentally, that the only time real analysis is done is when there are representative of the domain present in the room. Thus domain people and developers form an integrated team. Neither of these two communities is of any use without the other.
Using CRC cards to avoid getting bogged down:
There are two possible errors of judgement a team can make when developing a domain model, using class diagrams. One of them is getting bogged down in detail; the other is remaining at too high a level so that one’s feet barely touch ground. After a while getting bogged down is the greater risk, because the team really gets into what they’re doing. This can be bad if the big picture is lost and bad overall design decisions are made. One way of avoiding bogging down is to use CRC cards (see later in the notes).
Elaboration is concerned, then, with
· collecting further requirements (beyond inception)
· doing high level analysis and design to establish a base-line architecture and
· preparing the plan of construction.
It is, in particular, a phase that’s risk-driven. It involves writing code and constructing some of the use cases i.e. writing more code. This is in order that the plan for the construction phase can be put in place. You can’t plan the construction unless you do some constructing to make judgments as to whether your plan has any hope of being followed at least approximately.
Technological risk should be reduced by building a prototype of the system. Usually the dangers lie in the way the pieces of hardware and software are put together, rather than in the pieces themselves. One might know databases very well. One might know C++ well, but putting the two together can be surprisingly hard. Thus you put together a minimal application that accesses the database just to test that the database drivers work and so forth.
Skills risk can be reduced by a team getting the necessary training when it needs it rather than in one big chunk at the beginning, or none at all. When the training is really needed it will largely have been forgotten so it’s best delivered as you’re going along. Initially a consultant will work as part of the team, but later he/she’ll do more reviewing than doing. Also a group of developers should read widely and try to apply their reading. It has been quite common for many firms to embark on incremental/iterative projects with little or no training thus simply perpetuating the mess in which the software engineering community has so often found itself.
The elaboration phase can be regarded as finished when:
(a) all risks have been identified to the extent that you know how to handle them should any of them arise and
(b) that the time to build the already discovered use cases has been estimated to the nearest person week.
It should be remembered that not all the use cases will have been recognised or thought about during elaboration. There will still be some to discover during the construction phase. Remember too that some construction will have been done during the elaboration phase.
You can tell if someone doesn’t understand the spirit of iterative development, if he/she says: “Let’s work out all the use cases before we begin coding.” That person has allowed waterfall-type thinking to enter their head.
One might wonder why it’s not possible to pin down all the use cases, before coding. Well, it just doesn’t work in practice! The specification of a system changes as the system is being developed, not because of laziness of thinking, but because it just seems to be that way. It’s only the experience of seeing the partially running system that emerges bit by bit after each production-quality iteration that helps get the thoughts working towards a settled specification.
The Construction Phase
There is less of a distinction between the elaboration phase and the construction phase than might be apparent from these notes. Some people feel that construction is where the project moves from being in adaptive mode to predictive mode. Others don’t like that way of expressing it! It’s good that there is this debate; software process study is not a subject that’s static.
Planning time-boxed iterations
The construction phase should be divided into a number of time-boxed iterations, i.e. iterations which must meet their scheduled delivery dates. Not meeting your delivery date is illegal! A number of scenarios can be built for each iteration, or even a whole use case. If one finds that one is not going to get all the scenarios done for a particular release, one should slip a scenario into the next iteration, not postpone the release date of an iteration. If one finds one is regularly slipping in this way, one should change one’s plan as one goes along. One should expect to have to re-plan almost all the time. It’s important to build up a regular rhythm among team members; thus the iteration length should be fixed for all iterations.
Reverse Engineering:
Typically some member of the team will reverse engineer the previous iteration using a case tool to produce a set of UML diagrams that describe the code that was written for that iteration. This set of UML diagrams will be a lot more developed than the set of diagrams that was sketched out on white boards, and digitally photographed, to inspire and clarify the development of the code for that iteration.
Refactoring:
Each iteration should involve some re-working of existing code; this is not regarded as wasteful, but essential. One should be suspicious if any less than 10% of existing code is thrown away and replaced. That may seem strange, when one considers that each iteration is done to production quality. However, what’s happening is that refactoring is taking place. That’s a way of re-writing code to change its internal form without its external behaviour. It’s important not to use the refactoring procedure as an excuse to change the behaviour as that is foolishly attempting to do two things at once.
Planning an iterative project:
Work out the total effort (Qt) in person-weeks to construct the use cases that have already been discovered. Decide on your iteration length. Work out the effort per iteration, thus: