Shades of Scrum: Waterfall, Pipelining and Sprints
Waterfall and Scrum
We all know the traditional approach to software development – the sequential waterfall model.
Activities like analysis, design, implementation and test are done sequentially – an activity is completed before the next activity is started. The work is done by specialists like business analysts, software architects etc.
And there is Scrum: All activities are done in parallel in Sprints by a cross-functional team.
Combining Waterfall and Scrum
Moving from Waterfall to Scrum is a big step for most companies. Most companies are not capable of doing this change in a short period of time. Therefore a combination of Waterfall style phases and Scrum sprints is very common. Typically the Waterfall style phases embed the Scrum sprints.
For example market research and product definition would be done in Waterfall style. UI design, programming and component tests would be done in Scrum sprints. And then integration tests, roll-out and sales would again be done in Waterfall style.
The problem with this kind of combination of Waterfall and Scrum is not it used but that companies stay with it. A lot of companies that practice this combination of approaches find out that the advantages are limited:
- End-to-end responsibility is hard to establish due to the silos and hand-overs.
- Long time to market: Since the phases before and after the sprints stay the same the overall lead time often is reduced only by a few percentages.
- Long feedback cycle: The end-to-end feedback cycle is nearly as long as with the waterfall.
- Feedback may invalidate a large amount of work: If the last activity (e.g. sales) generates relevant feedback for the first activity (e.g. market research) a lot of work done during the activities in between may be invalidated.
Notice that Waterfall is not a bad idea in general. Especially when there are low risks and few things to lean, Waterfall may be a suitable approach.
Pipelining is a more beneficial approach. Pipelining simply means that several teams work in parallel sprints where one team generates the input for another team.
Pipelining is much better than the waterfall approach but in the end it just reduces batch sizes without changing the general way of working. Compared to Scrums all-in-one-team approach the drawbacks are the same as with the waterfall approach.
The first important aspect is that the feedback cycle is not one sprint but – in the case of three teams – three sprints.
The second aspect is that pipelining is a very poor approach to address learning.Pipelining assumes that the upstream team (team 1 in our example) hands over its output to the downstream team (team 2 in our example) and proceeds with the next sprint. But when the downstream team discovers something about the work done by the upstream team it is challenging to integrate that discovery into the work flow of the upstream team.
If for example a downstream team does testing, how would the upstream team handle the discovered bugs?
Fold the pipeline into Sprints
When you work with pipelining the whole thing can be folded into Sprints. When we have for example 3 teams doing pipelining with a Sprint length of 1 week it is easy to put all the participants in one team with a Sprint length of 3 weeks. The team then may decide to do some kind of pipelining but is empowered to modify the way of working within the sprints to suit current needs. And even if the team opts for pipelining it is more likely that the team members cooperate.
Scrum isn’t generally better than Waterfall. Both approaches have their specific pros and cons.
|Learning during the project||very expensive since a lot of work have to be redone||cheap|
|Innovation||unlikely by working within silos||likely due to cross-fertilization|
|Efficiency||efficient by using specialists for specialists tasks||less efficient since specialists work outside their specialization|
In general Scrum is better suited when a lot of things are unknown and therefore learning has to be incorporated in the project. In the end it boils down to the question what and how fast we need to learn – or where the risks are. When the roll-out of the software is of low risk it may be suitable to do it with pipelining or even Waterfall.
Therefore the question is not if Waterfall or Scrum is better suited. The question is, where we should use Waterfall and where we should use Scrum. And when we choose to use Waterfall pipelining is in most cases the better implementation – the smaller batch sizes (aka work in progress) generate shorter lead times.
- Shades of Scrum: Sprint length – the article describes some advantages of longer Sprint lengths (which may be the result of folding a pipeline into Sprints)
- Twelve emerging best practices for adding UX work to Agile development – In this article Jeff Patton suggests to use pipelining for User Experience (UX) work.