Waterfall Model
Iterative and Incremental Development

Iterative and Incremental Development

Iterative development was created as a response to inefficiencies and problems found in the waterfall model. Modified Waterfall, Rational Unified Process (RUP) and most, if not all, agile models are based on iterations.

General idea is to develop a system through iterations (repeated cycles) and incrementally (in small portions of time). Through them team members or stakeholders can learn from their mistakes and apply that knowledge on the next iteration.

Working through iterations means that the development of the application is split into smaller chunks. In each iteration features are defined, designed, developed and tested. Iteration cycles are repeated until fully functional software is ready to be delivered to production. The process does not try to start with the full set of requirements and design. Instead, team tries to prepare just what is needed for the successful delivery of the next iteration.

Some models have different names for iteration like sprint or time-boxed. Iterations can be limited in time; they end after the agreed period independently of the size of the scope that was done. Alternative way of doing iterations is to limit them in scope. They last until the agreed scope is fully finished (developed and tested).

It is a common practice that each iteration is finished with a demo to stakeholders. That demo is used as the learning process with the objective to correct the way next iteration is done or modify the scope. Since working model is available much earlier, it is much easier to spot problems before it is too late or too expensive to take corrective actions.

This way of developing is in stark contrast with the waterfall model where each phase of the software development life-cycle (SDLC) needs to be fully completed until the next one starts.

One of the main advantages of iterative development is that it allows more flexibility to adapt to changes. Unlike the waterfall model where unforeseen problems often surface late in the project and are very costly to fix, iterative approach, on the other hand, goes through short cycles that allow the team to learn, adapt and change the direction in the next iteration.

Why not everyone uses some form of iterative development?

Short answer to this question is that not everyone can use it effectively. Iterative development is much harder than the waterfall model. It requires higher level of technical excellence, more discipline and buyout from the whole team. It often requires that team members are capable of performing more than one type of tasks (for example develop and test or work on both front-end and back-end).

Changes need to be done across all roles when they come from the waterfall process. Two of those roles that are often most affected are integration engineers and testers.

Integration Engineers

Integration phase in iterative development is very short or, when done right, continuous. While in the waterfall model this phase can take even several weeks for bigger projects, iterations require it to be very short and done often. If, for example, testers need to test some functionality as soon as the code is done, integration and deployment needs to be almost instantaneous. There are many tools currently in use that facilitate the integration and deployment. Some of them are Puppet and Chef for configuration management and Jenkins, Hudson, Bamboo and Travis for Continuous Integration, Delivery and Deployment. Everything, or almost everything, should be scripted and run on certain events (commit to the repository or click of a button).

Testers

Testers (especially when used to only manual testing) are among those who have most difficulties adapting to the iterative process when they’re coming from the waterfall, especially if they are used to test the application after it is done. Switch to iterations forces them to act in a different way and think in forms of specific functionalities that should be verified instead of a fully developed system.

They need to work in parallel with developers in order to meet iteration deadlines.

Often there is no time to perform manual testing after the code of some specific functionality is finished. High level of automation is required. While developers are writing the code, testers need to write scripts that will verify functionalities that code will create. Automation requires certain coding skills that testers might not posses. As a result, test automation might be left to developers while testers continue being focused on manual testing (both are required to certain extent). However, in those cases testers might feel that part of their work and security it brings is taken away from them.

Advantages

End products are often more aligned to client needs due to abilities to demo functionalities done in each iteration and adjust depending on the feedback. Higher level of automation required for successful iterations allows faster detection of problems and creation of reliable and repeatable processes. That same automation, after initial investment, leads to reduction in costs and time to market. Interdependency among team members increases the shared knowledge within the team leading to a better understanding.

Some of those changes can be applied to the waterfall model but in many cases they are not. The incentives for doing them are not big since they might not be perceived as necessities. For example, Continuous Integration has big potential savings in non-waterfall projects due to the need to perform installations, deployments, testing and other tasks often and fast. In the waterfall model intention is to do the integration once (after the development phase is finished) so the investment for scripts and jobs that will perform repeatable and scheduled processes does not look like it provides enough return.

In most situations, iterative and incremental process contains the complexity and mitigates risks within a defined time box. This allows the team to continually review and adapt the solution according to the realities of the ever-changing situation.

Next post will explore in more depth Extreme Programming (XP) as one of the iterative models